This is the first of an occasional series of posts where we will look at a Content Assessment report and take a deep dive to review what information the report provides, what conclusions can be drawn from the information, and what likely actions you would take after reviewing the report. This series of posts will not cover how to run Content Assessment applications; instead, it will focus on using the results. For details on running Content Assessment, check out this post and this demo. This post will look at the SAS 9 Content Assessment Inventory.   Content Assessment Inventory   Inventory is, in many ways, the simplest of the Content Assessment reports. However, that does not make it any less important; it is the place you will likely start when reviewing Content Assessment results. The SAS Viya Inventory application scans SAS 9 systems and documents the type and volume of content. The scan includes the Metadata Server, file system areas and deployment and license information. Let’s look at the report.   What can we learn?   The initial page, “What is Being Used,” provides an overview of the content in the scanned 9.4 environment. It includes a high-level summary of object counts broken down by the SAS Software offering used. In addition, it includes a ranked frequency table listing all types of SAS 9.4 items found.     Select any image to see a larger version. Mobile users: To view the images, select the "Full" version at the bottom of the page.   Drill to counts gives detailed counts by object type segmented by portfolio and offering. In this tab, we can see unique counts for all objects in the SAS 9 deployment. In conjunction with the Overview tab, this tab can be used to determine the type and volume of content.       A few key facts from the report. In terms of SAS 9 objects, the environment has:   SAS Enterprise Guide Project SAS OLAP Cubes SAS Information Maps SAS Data Integration Studio Jobs SAS Stored Processes SAS BI Reports SAS Visual Analytics Reports Access Control Templates (ACTs) In 2024.09, SAS 9.4 Users and Groups are included in the report. The User and Group names are obfuscated, and a mapping file is provided to match the real name to the obfuscated name (\SAS9ContentAssessment\assessment\results\user_map_final.csv).   Drill to items gives us access to the detailed location of individual objects in the metadata or on the file system. In the example below we can see the name and location of the SAS Data Integration Studio Jobs. Drill to items allows you to filter by Portfolio and Sellable offering. It also allows a filter by storage type where you can subset to objects stored on the OS filesystem or in SAS Metadata.       Filesystem Object Info provides an inventory of file-system content in the areas scanned by the inventory tool. In addition to the location and type, the last modified and last accessed dates are provided, allowing you to identify if objects are being used.       The final two tabs of the report, Software Version and Hotfix Info and Software License Info provide details on the software installed and licensed in the SAS 9 environment.   To summarize from the Inventory report, we can learn what:   types of content and configuration are in the SAS 9 environment is the volume of each type of content is the location of the content in metadata or the file system file system content has been recently updated or used   What do we do with what we learned?   The inventory results can provide input to a house-cleaning process on the SAS 9.4 environment. You may identify objects no longer used in the environment that you do not want to migrate to SAS Viya. Removing these objects at the source can make the migration more efficient. Make a plan to clean up any unused content from the source environment. Note that Inventory does not indicate whether a metadata object is being used. Information on some of the profile reports and the Content Assessment Application Usage report can used to determine usage levels of content. You can also have Enterprise Session Monitor running against the environment for a period of time and check those results.   The results of Inventory should provide a road map for which Profile reports you should focus on when planning the migration. SAS 9 Profile takes the analysis a step further and drills deeper into the specific characteristics of each type of object (report, job, cube, etc) found. SAS Content Assessment Profile contains a series of reports for each SAS 9.4 content type. The profile reports will give you additional information about the specifics of each content type. The specifics will influence the effort needed to migrate to SAS Viya.   Using inventory results, check for items that do not automatically migrate to SAS Viya. Our inventory has a few Web Report Studio Reports we know do not migrate to SAS Viya. In addition, we identified Access Control Template usage where we know there are limitations in the migration process. You can check the results against the “Resources that Migrate” section of the SAS Viya Administration guide. For SAS employees and Partners, the Migration Summary document can be used to get detailed information on what is and is not supported in relation to migration to SAS Viya.   To summarize, the results of inventory can be used to:   clean the SAS 9 environment before a migration direct the next steps in the assessment process plan for items that will not migrate automatically to SAS Viya   Wrap Up   Inventory is the simplest of the SAS Content Assessment reports. Creating an inventory is an important part of the migration assessment process and can help direct the next steps in planning a successful migration from SAS 9 to SAS Viya. We started with the easy one in this series, but stay tuned for a look at some of the other Content Assessment Reports. For further information on Migration from SAS 9.4 to Viya, check out these resources:   Course: Migrating SAS 9.4 Content to the SAS Viya Platform on learn.sas.com Content Assessment Download SAS 9.4 to Viya Content Migration Documentation  SAS Demo | SAS Content Assessment Overview Migrating from SAS 9 to Viya: What's the first step Migration of Stored Processes to Jobs (update) Content Assessment delivers utility applications to support SAS Viya Content Migration Content Assessment: SAS 9 Code Check for Internationalization Go with the flow; migrating Enterprise Guide Projects to SAS Studio Flows SAS Viya 2021.2 – Migrating SAS Data Integration (DI) Studio Jobs to SAS Studio Flows A first look at Migration from SAS 9.4 to Viya 4 SAS Demo | SAS Viya Migration of Enterprise Guide Project to SAS Studio Flows – YouTube SAS Demo | Using the modifySASCode Content Assessment Application – YouTube SAS Demo | Migrating SAS Data Integration Studio Jobs to SAS Studio Flows     Find more articles from SAS Global Enablement and Learning here. ... View more

In the GEL SAS Viya administration class, we finish the week with a troubleshooting section. In this section, we create problems in a SAS Viya deployment and ask students to fix them. In this post, I will cover the basic troubleshooting steps we follow. Following a general process for identifying and resolving issues can facilitate effective troubleshooting. ​   The diagram below shows a generic SAS Viya issue identification and resolution method. Initially, we capture the message from the interface or relevant details from the problem process’s logs. The next step involves identifying the servers and services contributing to the problem, which can be challenging without a solid understanding of SAS Viya's architecture. For example, if a report fails to open, potential causes could include issues with report configurations, authorization settings, identity management, or data-related problems. ​ ​   After pinpointing the servers and services involved, the next task is to collect additional information. Examining messages preceding and following the error is crucial to gain context. We adjust logging levels for affected services to replicate the issue and gather more diagnostic data if necessary. Once sufficient information is collected to address the problem and the problem is fixed, it's important to reset logging levels to defaults to prevent system performance degradation. ​ Select any image to see a larger version. Mobile users: To view the images, select the "Full" version at the bottom of the page.   This process is fairly generic and could be used for any software problem. Let's examine the specifics of collecting more information about a problem in SAS Viya running on Kubernetes.   Troubleshooting issues with SAS Viya often involves examining Kubernetes resources using the Kubernetes command-line tool kubectl or a graphical tool like Lens. The diagram below shows key kubectl commands to identify the Viya services with the problem and collect more information. These will provide the basic information to start debugging a problem. The kubectl get, describe, and logs commands are shown here for pods but can be used for other Kubernetes resources. For example, you could also do a kubectl get pvc or a kubectl describe job. Let's look at what information you can get from these commands.​     SAS Viya services run in Kubernetes PODS. One method to identify which PODS to focus on when researching a problem is to use the SAS Viya readiness service. The readiness service checks the status of the SAS Viya platform to determine whether it is ready for use. The service performs all of its checks every 30 seconds. After the software is deployed, the service can be consulted to determine if the deployment is ready for use or if there are PODS with issues. To do that, you can check the readiness POD status and for more detailed information, you can use the to identify services with problems.   Kubectl get   The first step in debugging is often to perform a kubectl get pods for all or some of the PODS in the SAS Viya deployment. The kubectl get command lists Kubernetes resources and their status. The output below shows a partial screenshot of a get pods on a Viya deployment.     Let's look at what information we can get from the output. The Status column of the output shows the POD status. Possible POD status values are:   Pending: pod has been accepted by the Kubernetes cluster, but one or more of the containers has not been set up and made ready to run​ Running: pod has been bound to a node, and all of the containers have been created. At least one container is still running or is in the process of starting or restarting.​ Succeeded: All containers in the Pod have been terminated successfully and will not be restarted (the status for a POD that is part of a Job) Failed: All containers in the Pod have terminated, and at least one container has terminated in failure​   PODS with issues may have a status of Failed or Pending. Running is what you would expect from a functioning POD; however, it is important to note that a pod that is in a running state does not necessarily mean it is functioning properly. In the get pods output, the Ready column shows two numbers for each POD. The second is the total number of containers in a POD, and the first is the number of ready containers in the POD. For example, 1/1 indicates that all containers in the Pod are ready and running, whereas 0/3 indicates that none of the 3 containers in the POD are running.   We can see two examples of this in the output above where sas-cas-server-default-controller has a 0/3 and sas-cas-control has a 0/1. A status other than 1/1 or 2/2 can indicate a problem. Notice the status of sas-cas-server-default-controller shows Init:0/2. In Kubernetes, the status Init:0/2 for a pod indicates that the pod has two init containers, and neither of them has been completed successfully yet. (Init containers perform startup tasks in PODS, which must be completed before the main application containers start). The output here indicates there is a problem starting the CAS server. You can also use labels or the grep command to get the status of specific pods. Here are some examples of targeted get commands   Get the SAS Folders pod kubectl get pods -l app=sas-folders Get all PODS with mining in the name kubectl get pods | grep mining​   Kubectl describe   Once you have identified where the problem is, you can use the kubectl describe command to get more details about a POD. This command provides detailed information about Kubernetes resources, including overall status,  status of individual containers, configuration, labels, and resource requests. The event history of the POD (at the bottom of the output) is particularly useful for debugging because it displays recent events and problems encountered during the resource’s lifecycle, which can help identify issues.  This example does a kubectl describe of the CAS controller POD.   kubectl describe pod sas-cas-server-default-controller     Notice the event history shows a failure to mount a directory from an NFS server.   Kubectl logs   Of course, logs are a primary source of information for identifying and debugging problems. The kubectl logs command lets you access logs from your resources on a per-container basis or in aggregate. You can view a snapshot of currently collected logs, continually stream new log lines to your terminal, and access historical log lines emitted by terminated containers. ​Useful options on the logs command are: -f stream to the logs, -c to select a specific container, --tail to specify the number of lines to view at the end of the log and --since to specify a time period. Since have a problem with sas-cas-control, let’s look at its logs with the kubectl logs command using the label to identify the current POD.   kubectl logs -l app=sas-cas-control | gel_log     To debug the problem, we may need to raise log levels on Viya services. The process for adjusting log levels depends on the type of service.​ To adjust logging levels for most services, we can make changes in the SAS Environment Manager within the configuration area. For example, if we have issues with the identities service, we can set the logging level to "debug" to gather more detailed information. However, the process is slightly different when dealing with CAS and SAS Programming run-time. We still make the changes in the SAS Environment Manager, but we need to edit the XML for the specific service. If we do change the logging levels should be sure to reset the log level when we are done.   In our example, the problem is clear. The log shows that sas-cas-control is waiting for the CAS server to be ready. The two issues are related: the CAS server cannot start, and sas-cas-control is waiting for a CAS server. The root cause of this issue, where the CAS server cannot mount the NFS server directory, could be various. It could be networking issues, permissions on the mounted drive, etc.   kubectl events   What if we don't get a log? This can happen when containers in a POD fail to start or keep restarting.  This makes it challenging to debug the issue. In these scenarios, you can use the kubectl events command to gather useful information. The kubectl events command returns information from the Kubernetes event log for the namespace. When Kubernetes components such as nodes, pods, or containers change state, they automatically generate events to document the change. Events provide key information about the health and status of your cluster. They inform you if container creations are failing and pods are being rescheduled. Information in events can help you troubleshoot issues. In this command, we look for messages relating to CAS using kubectl events   kubectl get events | grep sas-cas     The kubectl events command confirms the issue is with mounting a directory for the volume sas-viya-gelcorp-volume. Notice the sas-readiness POD is also mentioned. If we look at the readiness POD log, we can see that the services that are not ready all depend on a CAS server.   kubectl logs -l app=sas-readiness | gel_log     Another advanced but useful command we could use to further our troubleshooting steps is kubectl exec. The exec command lets you start a shell session inside a container. Once inside the container, you can perform tasks like inspecting the container’s file system, checking the state of the environment, and performing advanced debugging steps. Accessing the container in a shell window can be useful when logs alone don't provide enough information. In our problem with accessing the NFS server, we could further debug it by execing into the POD and running the mount command to collect more information on the failure.   Wrap Up   That is a high-level review of some useful troubleshooting steps in SAS Viya 4. In this post, we have focused on using the kubectl to collect information. Of course, you could also use SAS Viya Monitoring for Kubernetes, an open-source solution provided by SAS that supports issue detection, alerting, and investigation. It includes a logging area where users can search, filter, chart, and display log messages and a monitoring and alerting area to collect, display, and analyze metric data and manage alerts.​   If you want to test your troubleshooting skills, check out the last chapter of the GEL SAS Viya Administration class. Finally, the SAS Viya documentation has some great content about troubleshooting, I have included links below.   Documentation: SAS Viya Platform: Troubleshooting Documentation: Readiness Service Documentation: Logging   Feel free to add any tips and tricks you have found useful in this post's comments. have a problem with sas-cas-control, let’s look at its logs with the kubectl logs command using the label to identify the current POD.     Find more articles from SAS Global Enablement and Learning here. ... View more

With the 2024.03 release of SAS Viya the sas-viya command-line interface is available for download in a container image. This new feature will be handy for automating SAS Viya administrative tasks. This is a topic I have addressed in the past where we built our own container image containing the sas-viya CLI. In this post, I will provide an overview of the new SAS-delivered sas-viya-cli container image and review how you can install it and use it to perform SAS Viya administration tasks.   Getting the sas-viya-cli container image To use the sas-viya-cli in a container we need to pull it from the container registry. The sas-viya CLI Container image is available with an SAS Viya license.  To pull the container image SAS Mirror Manager is used to identify the latest image in the SAS Container registry and to return the credentials to login into the registry. In this step, passing the certificates from our Viya order, we use mirror manager to return the image name of the sas-viya cli image in our order. The name of the image in the registry contains the string sas-viya-cli. In this step we store the name in the environment variable ${climage} for later use climage=$(mirrormgr list remote docker tags --deployment-data /home/cloud-user/project/deploy/license/SASViyaV4_${GELLOW_ORDER}_certs.zip --cadence stable-2024.03 | grep sas-viya-cli) echo Order Number is ${GELLOW_ORDER} and latest image is ${climage} Output: Order Number is 9CYNLY and latest image is cr.sas.com/viya-4-x64_oci_linux_2-docker/sas-viya-cli:1.1.0-20240319.1710834807264   To pull the images we need to authenticate to the docker registry. In this step, we use mirror manager to retrieve the login credentials and log in to the docker registry. logincmd=$(mirrormgr list remote docker login --deployment-data /home/cloud-user/project/deploy/license/SASViyaV4_${GELLOW_ORDER}_certs.zip) echo $logincmd eval $logincmd   Output: docker login -u 9CYNLY -p '!|gd^X3Vq0fVJbiL1h9N1JVJ#0mqf986' cr.sas.com WARNING! Using --password via the CLI is insecure. Use --password-stdin. WARNING! Your password will be stored unencrypted in /home/cloud-user/.docker/config.json. Configure a credential helper to remove this warning. See https://docs.docker.com/engine/reference/commandline/login/#credentials-store Login Succeeded   Once we are authenticated to the docker registry we can pull the sas-viya-cli image and tag the pulled image with an easy-to-use name.   docker pull ${climage} docker tag ${climage} sas-viya-cli:v1   Now we can address the image with the simplified name and test that it works. To do that we will do a docker run of the image and, for test purposes pass the --help command to the CLI. By default, the container image has an entry point, that entry point will run the command sas-viya. docker container run -it sas-viya-cli:v1 --help   Authenticate to SAS Viya   Now that we have the cli image we can use it for the same Viya administration tasks that we use for the standard sas-viya cli. The same as the sas-viya CL to authenticate we need the .certs file from the SAS Viya environment. In this step copy the certificates file to the /tmp directory. kubectl cp $(kubectl get pod | grep "sas-logon-app" | head -1 | awk -F" " '{print $1}'):security/trustedcerts.pem /tmp/trustedcerts.pem   The containerized CLI gives us a few different methods to authenticate. The simplest method is to pass the SAS Viya endpoint, the username, and the password in environment variables. In the step below, we use the docker run command to authenticate as sasadm and run sas-viya identities whoami. Notice that to make the certificates file available, we also mount the local /tmp directory to the /security directory inside the container. export VIYA_USER=sasadm export VIYA_PASSWORD=lnxsas export SAS_SERVICES_ENDPOINT=https://gelcorp.pdcesx02193.race.sas.com docker run -it -e SAS_SERVICES_ENDPOINT -v /tmp:/security -e VIYA_USER -e VIYA_PASSWORD sas-viya-cli:v1 --output text identities whoami   Output: https://gelcorp.pdcesx02193.race.sas.com Login succeeded. Token saved. Id sasadm Name SAS Administrator Title EmailAddresses [map[value:sasadm@gelcorp.com]] PhoneNumbers Addresses [map[country: locality:Cary postalCode: region:]] State active ProviderId ldap CreationTimeStamp ModifiedTimeStamp   NOTE: in this syntax, you must use the export command to set the environment variables. An alternative approach shown below is to use the -e VAR=VALUE.   MY_USER=sasadm MY_PASSWORD=lnxsas export SAS_SERVICES_ENDPOINT=https://gelcorp.pdcesx02193.race.sas.com docker run -it -e SAS_SERVICES_ENDPOINT -v /tmp:/security -e VIYA_USER=${MY_USER} -e VIYA_PASSWORD=${MY_PASSWORD} sas-viya-cli:v1 --output text identities whoami   When you use this approach the sas-viya cli, running inside the container, will authenticate to SAS Viya on every execution. This is achieved inside the container by running an entry-point script that authenticates. The entry point also runs the sas-viya command, meaning that to execute a plugin command, you pass after the container name:   any options for the sas-viya cli e.g --output text the name of the CLI plug-in to run e.g identities the plug-in command e.g. whoami   The second method to authenticate is to pass a JWT token. For this method to work, you must NOT have the environment variables VIYA_USER and VIYA_PASSWORD set. But you should set the environment variable JWT_TOKEN to the value of a JWT_TOKEN. If you authenticate outside the container with the sas-viya cli a credentials file is created at ~/.sas/credentials.json. The credentials file contains the JWT_TOKEN and a refresh token. The JWT token is in the “access_token” field and can be extracted from this file. In the code below, we use a Python program, savetoken.py to do this (you can use any tool of your choice). In the example below, the JWT_TOKEN was created when the geladm user authenticated.   unset VIYA_USER unset VIYA_PASSWORD /opt/pyviyatools/savetoken.py export SAS_SERVICES_ENDPOINT=https://gelcorp.pdcesx02193.race.sas.com export JWT_TOKEN=$(cat /home/cloud-user/.sas/gelcorp_token.txt ) docker run -it -e SAS_SERVICES_ENDPOINT -v /tmp:/security -e JWT_TOKEN sas-viya-cli:v1 --output text identities whoami   Output: Login succeeded. Token saved. Id geladm Name geladm Title Platform Administrator EmailAddresses [map[value:geladm@gelcorp.com]] PhoneNumbers Addresses [map[country: locality:Cary postalCode: region:]] State active ProviderId ldap CreationTimeStamp ModifiedTimeStamp   Use the sas-viya-cli contaner Image   There are three main ways we can use the containerized version of the sas-viya CLI.   Run commands and retrieve output   We can run ad-hoc CLI commands using the docker run command and return the command’s output. This is an approach we have already seen in prior examples. Here, we list the current folders in the SAS Viya Environment. Each execution of the docker run commands starts a container, authenticates to SAS Viya inside the container and runs a SAS Viya plugin and command. The plugin and command are passed after the image name on the docker run command. You do not need to specify sas-viya. docker run -it -e SAS_SERVICES_ENDPOINT -v /tmp:/security -e VIYA_USER -e VIYA_PASSWORD sas-viya-cli:v1 --output text folders list-members --path / --tree   Output:   https://gelcorp.pdcesx02193.race.sas.com Login succeeded. Token saved. |—— Public |—— Products | |—— SAS Studio Flow | |—— SAS Environment Manager | |—— SAS Visual Machine Learning | |—— SAS Visual Analytics | |—— SAS Studio | |—— SAS Analytics Common Services |—— Users |—— Model Repositories | |—— DMRepository | |—— Public Output:   Run commands from inside the container   Run sas-viya CLI commands in interactive mode from inside of the container. If you do not add a CLI command after the container name, the docker run command will open a shell prompt inside the container. You can run ad-hoc CLI commands in this shell window. In this example, we start the session, run two report commands inside the container, and then exit the container. docker run -it -e SAS_SERVICES_ENDPOINT -v /tmp:/security -e VIYA_USER -e VIYA_PASSWORD sas-viya-cli:v1     There is an additional step if your sas-viya command creates output on the filesystem or you need to accept input from the local filesystem. The file system inside the container is ephemeral by default. That means that if we want to persist file-system output from a CLI command, we need to mount a location from the local filesystem into the container. In the example below, using -v /tmp/cli-output:/tmp we mount the local directory /tmp/cli-output to /tmp inside the container. The command below outputs a JSON file with the SAS Viya identities configuration. If we output that file to /tmp, it will be available at /tmp/cli-output on the local filesystem.     docker run -it -e SAS_SERVICES_ENDPOINT -v /tmp:/security -v /tmp/cli-output:/tmp -e VIYA_USER -e VIYA_PASSWORD sas-viya-cli:v1 configuration configurations download -s sas.identities --target /tmp/identites.json   Output   https://gelcorp.pdcesx02193.race.sas.com Login succeeded. Token saved. Write is complete to "/tmp/identites.json".     NOTE: By default, the CLI runs as the user with the UID 1001. To output files, you must make sure that this user has write access to the local file system directory.   Build your own image   You can build your own docker image based on the SAS-provided sas-viya cli image. This allows you to add tools to the image and customize it to your needs. The docker file below shows the definition of a new sas-viya-cli image.   FROM registry.unx.sas.com/convoy/sas-viya-cli USER 0 #install required software RUN dnf -y install git \ dnf -y install python3-pip \ pip3 install requests configobj\ ln -sfn /usr/bin/python3.6 /usr/bin/python \ dnf clean all # install pyviyatools RUN cd / ; git clone https://github.com/sassoftware/pyviyatools.git; /pyviyatools/setup.py RUN ln -s /opt/sas/viya/home/bin/sas-viya /usr/bin/sas-viya USER 1000 ENTRYPOINT [ "" ] Use the file to build the docker image. docker build . -f sas-viya-cli.Dockerfile -t sas-viya-cli --no-cache   With the updated image, we:   override the entry point so that the default authentication is not used specify the user that the container runs as mount the local CLI configuration and credentials file into the container specify the CLI profile to use as an environment variable   In the command below we run ad-hoc CLI processing using the new container image. The certs file and the CLI configuration and credentials file are mounted into the container and the SAS_CLI_PROFLE environment variable is set because we overrode the entry point we do need to execute the sas-viya part of the command.   docker container run -it \ -v ${SSL_CERT_FILE}:/cli-home/.certs/`basename ${SSL_CERT_FILE}` \ -v ~/.sas/config.json:/cli-home/.sas/config.json \ -v ~/.sas/credentials.json:/cli-home/.sas/credentials.json \ -e SSL_CERT_FILE=/cli-home/.certs/`basename ${SSL_CERT_FILE}` \ -e REQUESTS_CA_BUNDLE=/cli-home/.certs/`basename ${SSL_CERT_FILE}` \ -e SAS_CLI_PROFILE=default \ sas-viya-cli:latest sas-viya --output text identities whoami   This is a way to easily use different CLI profiles and re-use existing credentials files. The token in the CLI credentials file is valid for 1 hour, but the provided refresh token allows the token to be renewed automatically for 14 days. Wrap-up   The new sas-viya-cli container image is a great addition to the toolbox for SAS Administrators and others who want to automate tasks in SAS Viya. It has all the benefits of containerization discussed in my previous post like standardized packaging, version control, consistency and scalability. My colleague David Esterich has already started to use it in his Azure DevOps work. Please check out his posts linked below. In addition, the installation and usage of the container is covered in the SAS Viya Administration guide. For more information and related posts check out these resources:   SAS Viya Administration Guide: Run the CLI from a Container SAS Viya Command-Line Interface for Administration (update) Introducing New SAS Viya CLI container image in Azure DevOps (March 2024 Update) Streamlining SAS Viya Authentication in Azure DevOps SAS administration command-line interface in a container: Part 1 Docker Using Apache Airflow to automate SAS Viya administration tasks Install and use the SAS Viya CLI on Windows Automating post-deployment Viya tasks with REST and the administration CLI       ... View more

PostgreSQL is an open-source database management system that SAS provides uses as the SAS Infrastructure Data Server. The SAS Viya platform uses this component to store user content. The SAS Viya platform can use either an internal instance of PostgreSQL that it provides or an external instance. The PostgreSQL data transfer tool allows you to switch between these two options. This post demonstrates the process of switching a Viya deployment from an internal PostgreSQL instance to an external instance using that Azure Flexible PostgreSQL server.   The process we will follow contains the following steps.   Perform Prerequisites Prepare to use the PostgreSQL Data Transfer Tool Transfer data using the PostgreSQL Data Transfer Tool Switch SAS Viya to point to External PostgreSQL Validate that that switch worked Delete internal PostgreSQL.   This process is documented in the SAS Viya Operations guide. Please check out the documentation for the specifics of each cloud provider and offering. In this post, we will demonstrate the process for the SAS Infrastructure Data Server with SAS Viya Visual Analytics on Azure.   Perform prerequisites and prepare to use the PostgreSQL Data Transfer Tool   If you deployed with the internal instance of PostgreSQL it uses Crunchy PostgreSQL. Two versions of Crunchy PostgreSQL are used by SAS Viya. The steps in the process vary based on the version used in the deployment. Run the following code to get your software release to determine what version of Crunchy PostgreSQL is being used in your SAS Viya deployment. SAS Viya 2022.10 and later use Crunchy PostgreSQL 5, prior releases use version 4. The output below shows that we are using Crunchy version 5. The rest of the process here applies to version 5. If you have an earlier version of Crunchy check out the documentation.   kubectl get cm -o yaml | grep 'SAS_CADENCE_DISPLAY_NAME'   Expected output:   SAS_CADENCE_DISPLAY_NAME: Stable 2024.01 {"apiVersion":"v1","data":{"SAS_BUILD_TYPE":"x64_oci_linux_2-docker","SAS_CADENCE_DISPLAY_NAME":"Stable 2024.01",....   It is recommended that we have a database administrator user in the target instance of the same name as the current SAS Infrastructure Data Server administrator. The default for SAS Viya is dbmsowner. To check that this is the name on the current deployment run the following code:   kubectl get secrets sas-crunchy-platform-postgres-pguser-dbmsowner -o template={{'.data.user'}} | base64 -d   Expected output:   dbmsowner   Some of the upcoming steps will be performed from inside the crunchy PostgreSQL primary POD. To get the primary POD name and exec into the POD run the following:   export SOURCE_CLUSTER=sas-crunchy-platform-postgres PRIMARY=$(kubectl get pod --selector="postgres-operator.crunchydata.com/role=master,postgres-operator.crunchydata.com/cluster=$SOURCE_CLUSTER" -o jsonpath='{.items[*].metadata.name}' | awk '{print $1}') echo Primary Crunchy POD is $PRIMARY kubectl exec -it $PRIMARY -c database -- bash   Expected output:   Primary Crunchy POD is sas-crunchy-platform-postgres-00-c992-0   The next few steps will be performed from within the Crunchy PostgreSQL primary POD. Firstly, let us check the size of the existing database. To account for the future growth of your PostgreSQL database, SAS recommends that your new external instance has twice the space that your existing internal instance uses.  To determine the size of the existing PostgreSQL database from within the crunchy primary POD run this code:   psql -c "select pg_size_pretty ( pg_database_size('SharedServices'));"   Expected output:   pg_size_pretty ---------------- 377 MB (1 row)   The external instance of PostgreSQL must meet certain requirements. In addition to an admin user matching the name of the current user, the instance must have the appropriate extensions enabled. In the following steps, we will update the configuration of External PostgreSQL. To perform these steps, we need to know the connection information for the Azure Flexible PostgreSQL server. Using that connection information, we will connect to the target instance from inside the primary Crunchy PostgreSQL POD. The connection information is stored in the following environment variables.   TARGET_HOST_NAME="sasgnn-p21077-default-flexpsql.postgres.database.azure.com" TARGET_HOST_PORT="5432" TARGET_USER_NAME="pgadmin" TARGET_USER_PASSWORD="Aa0EmhCxZJYOJWBMdEf3uj1zr60" DATASERVER_NAME="sas-platform-postgres"   Firstly, let's create the dbmsowner user in the Azure Flexible PostgreSQL server instance and allocate the user the required privileges.   PGPASSWORD=$TARGET_USER_PASSWORD psql -h $TARGET_HOST_NAME -p $TARGET_HOST_PORT -U $TARGET_USER_NAME -d postgres -c "CREATE USER dbmsowner" PGPASSWORD=$TARGET_USER_PASSWORD psql -h $TARGET_HOST_NAME -p $TARGET_HOST_PORT -U $TARGET_USER_NAME -d postgres -c "ALTER USER dbmsowner CREATEDB" PGPASSWORD=$TARGET_USER_PASSWORD psql -h $TARGET_HOST_NAME -p $TARGET_HOST_PORT -U $TARGET_USER_NAME -d postgres -c "ALTER USER dbmsowner CREATEROLE" PGPASSWORD=$TARGET_USER_PASSWORD psql -h $TARGET_HOST_NAME -p $TARGET_HOST_PORT -U $TARGET_USER_NAME -d postgres -c "ALTER USER dbmsowner WITH PASSWORD '$TARGET_USER_PASSWORD';" PGPASSWORD=$TARGET_USER_PASSWORD psql -h $TARGET_HOST_NAME -p $TARGET_HOST_PORT -U $TARGET_USER_NAME -d postgres -c "GRANT CONNECT ON DATABASE postgres TO dbmsowner;" PGPASSWORD=$TARGET_USER_PASSWORD psql -h $TARGET_HOST_NAME -p $TARGET_HOST_PORT -U $TARGET_USER_NAME -d postgres -c "GRANT azure_pg_admin TO dbmsowner;"   Now let's enable the extensions on the Azure PostgreSQL flexible server instance. In this step, we use the Azure CLI to enable these extensions and then restart the database server.   az account set --subscription ${SUBSCRIPTIONID} az postgres flexible-server parameter set --resource-group ${RG} --server-name sasgnn-p21077-default-flexpsql --name azure.extensions --value LTREE,PG_CRON,PG_STAT_STATEMENTS,PG_TRGM,PGAUDIT,PGCRYPTO,PLPGSQL az postgres flexible-server parameter set --resource-group ${RG} --server-name sasgnn-p21077-default-flexpsql --name shared_preload_libraries --value PG_CRON,PG_STAT_STATEMENTS,PGAUDIT az postgres flexible-server restart --resource-group ${RG} --name sasgnn-p21077-default-flexpsql --subscription ${SUBSCRIPTIONID}   The target instance is now ready for data transfer. It is recommended that you perform a SAS Viya Backup before continuing with the process.   Prepare and then Transfer Data using the PostgreSQL Data Transfer Tool   The PostgreSQL Data Transfer Tool is available for download from support.sas.com. The tool is delivered in a ZIP file and is installed by simply unzipping it to a directory of your choice. The tool is implemented as a Kubernetes job and the job is configured by updating the provided configuration files and manifests.   SAS provides a script to determine the values of the required variables and insert the values into the manifest files of the tool. The script requires that mirror manager is deployed and that you have access to the license files in the SAS Viya deployment assets. In this step run the script to see the required values.   NOTE: if you have CDS PostgreSQL data you need to perform these steps for that database also.   cd /home/cloud-user/project/deploy/pgtransfer/scripts/ ./manifest-values.sh --namespace gelcorp --mirrormgr /opt/mirrormgr/mirrormgr --deployment-data /home/cloud-user/project/deploy/license/SASViyaV4_${GELLOW_ORDER}_certs.zip   Expected output:   Token values SAS-CERTFRAME-IMAGE-NAME = cr.sas.com/viya-4-x64_oci_linux_2-docker/sas-certframe:3.49.1-20240105.1704477691165 SAS-CUSTOMER-PROVIDED-CA-CERTIFICATES-CONFIGMAP-NAME = sas-customer-provided-ca-certificates-6ct58987ht SAS-IMAGE-PULL-SECRETS-NAME = sas-image-pull-secrets-b7bt458c8f SAS-INGRESS-CERTIFICATE-SECRET-NAME = sas-ingress-certificate SAS-POSTGRES-DATA-TRANSFER-IMAGE-NAME = cr.sas.com/viya-4-x64_oci_linux_2-docker/sas-postgres-data-transfer:1.6.0-20240201.1706816594189 STORAGE-CAPACITY (cds) = STORAGE-CAPACITY (platform) = 256Gi   If we are happy with the required values, we can run the script using the --apply parameter to update the PostgreSQL Data Transfer Too manifests and insert the required values.   cd /home/cloud-user/project/deploy/pgtransfer/scripts/ ./manifest-values.sh --namespace gelcorp --mirrormgr /opt/mirrormgr/mirrormgr --deployment-data /home/cloud-user/project/deploy/license/SASViyaV4_${GELLOW_ORDER}_certs.zip --apply   Expected output:   INFO: Skipping substitutions for cds - it was not found in the SAS Viya platform deployment. INFO: Applying changes to '/home/cloud-user/project/deploy/pgtransfer/manifests/platform/pvc.yaml'. INFO: Applying changes to '/home/cloud-user/project/deploy/pgtransfer/manifests/platform/job.yaml'.   Before running the tool to transfer data populate the user_input.yaml. The user_input.yaml is located in the install directory of the tool at /manifests/platform/user_input.yaml. The key information in this file is the connection information to the target database server.   The tool can also be run in phases using the "run" options.   For example, if you want to only run the pre-check steps to check that the target instance meets the requirements for transfer set run.pre-check to true and the other two run variables to false. If any of the run phases are marked false, then in order to run that phase later, you must delete the job, modify the user_input.yaml file, and re-run the Job.   target: database: "postgres" admin: "dbmsowner" password: "${TARGET_USER_PASSWORD}" fqdn: sasgnn-p21077-default-flexpsql.postgres.database.azure.com port: 5432 sslmode: verify-full run: pre-check: true transfer: true post-check: true overwrite: false status_update_timespan: 15m00s sleep_after_completion: 0m00s log_level: info   When you are ready to transfer data, you must stop the SAS Viya environment.  The following code will kick off the SAS Viya stop job and wait for the PODS to terminate.   kubectl create job sas-stop-all-`date +%s` --from cronjobs/sas-stop-all RUNNING_PODS=$(kubectl get pods --field-selector=status.phase=Running -o go-template='{{len .items}}') until [ $RUNNING_PODS -lt 2 ]; do \ sleep 5; echo Running Pods: $RUNNING_PODS; \ RUNNING_PODS=$(kubectl get pods --field-selector=status.phase=Running -o go-template='{{len .items}}'); \ done   Expected output:   Running Pods: 132 Running Pods: 87 Running Pods: 65 Running Pods: 31 Running Pods: 9 Running Pods: 4   Now we can apply the manifests to run the data transfer job. The PostgreSQL Data Transfer Tool uses two standard PostgreSQL utility programs: pg_dump to back up data from your source instance and pg_restore to load the backup to your target instance. During execution, the transfer tool performs the following tasks:   Starts the internal PostgreSQL server if it is not already running. Performs a logical backup of each database in the source server. Restores the logical backup into the target external PostgreSQL cluster, creating or re-creating the databases as necessary.   To start the transfer job, apply the mainfests.   cd /home/cloud-user/project/deploy/pgtransfer kustomize build manifests/platform -o platform-transfer.yaml kubectl apply -f platform-transfer.yaml   Expected output:   serviceaccount/sas-platform-postgres-data-transfer-sa created role.rbac.authorization.k8s.io/sas-platform-postgres-data-transfer-role created rolebinding.rbac.authorization.k8s.io/sas-platform-postgres-data-transfer-rolebinding created secret/sas-platform-postgres-data-transfer-input-f7k8hm7khk created persistentvolumeclaim/sas-platform-postgres-data-transfer-backups created job.batch/sas-platform-postgres-data-transfer created   To check the success of the job, review the log. The log should contain messages indicating the successful execution of the data transfer tool. You can grep for specific text like "successfully copied database".   kubectl logs job/sas-platform-postgres-data-transfer   Expected output:   status=success duration=515ms level=0 logger=summary ts=2023-10-05T15:47:55.015215518Z msg="pre-check sasgnn-p21077-default-flexpsql.postgres.database.azure.com | check server versions" status=success duration=15.3ms level=0 logger=summary ts=2023-10-05T15:47:55.015274819Z msg="pre-check sasgnn-p21077-default-flexpsql.postgres.database.azure.com | check shared preload libraries" status=success duration=3.95ms matchingEssentialSharedPreloadLibraries=2 level=0 logger=summary ts=2023-10-05T15:47:55.015286719Z msg="pre-check sasgnn-p21077-default-flexpsql.postgres.database.azure.com | check extensions availability" status=success duration=381ms level=0 logger=summary ts=2023-10-05T15:47:55.015295719Z msg="transfer sas-crunchy-platform-postgres" status=success duration=1m3.2s level=0 logger=summary ts=2023-10-05T15:47:55.015325319Z msg="transfer sas-crunchy-platform-postgres | backup SharedServices to /backups/sas-crunchy-platform-postgres/SharedServices" status=success duration=36.7s level=0 logger=summary ts=2023-10-05T15:47:55.015372120Z msg="transfer sas-crunchy-platform-postgres | restore from /backups/sas-crunchy-platform-postgres/SharedServices" status=success duration=26s level=0 logger=summary ts=2023-10-05T15:47:55.015394720Z msg="post-check SharedServices" status=success duration=14.3s verifiedDatabaseAttributes=3 totalVerifiedRoleDetails=5789 verifiedRoles=1 totalVerifiedTableColumns=6840 totalVerifiedTableIndexes=1027 tablesWithMatchingNumbersOfRows=821 tablesWithMatchingNumbersOfConstraints=821 totalVerifiedTables=821 verifiedSchemas=84   NOTE: If the transfer fails you can delete the job using kubectl delete job/sas-platform-postgres-data-transfer and run it again using kubectl apply.   Switch SAS Viya to point to External PostgreSQL   The data has been copied to the external PostgreSQL instance, but SAS Viya is still using the internal instance. Once we are sure that the data has been transferred successfully, we can switch the SAS Viya deployment over to use the new external PostgreSQL instance. Follow the external PostgreSQL Configuration steps here to do this, do not remove the entries for your internal PostgreSQL clusters yet. They are removed later when the internal cluster is deleted. To summarize the process.   Copy /sas-bases/examples/postgres/postgres-user.env to $deploy/site-config/postgres/platform-postgres-user.env  and add the PostgreSQL database user (dbmsowner) and password. Add a secret generator that refers to the file to. the secretGenerator block of the kustomization.yaml Copy the /sas-bases/examples/postgres/dataserver-transformer.yaml/dataserver-transformer.yaml from to the $deploy/site-config/postgres/ and edit the file to update the values. Add the file to the transformers section of the kustomization.yaml.  Here is an example of the of the dataserver-transformer.yaml.   apiVersion: builtin kind: PatchTransformer metadata: name: sas-platform-postgres-dataserver-transformer patch: |- - op: replace path: /spec/users/0/credentials/input value: secretRef: name: platform-postgres-user usernameKey: username # For internal use, do not modify passwordKey: password # For internal use, do not modify - op: replace path: /spec/registrations/0/host value: sasgnn-p21077-default-flexpsql.postgres.database.azure.com - op: replace path: /spec/registrations/0/port value: 5432 - op: replace path: /spec/ssl value: true target: group: webinfdsvr.sas.com kind: DataServer version: v1beta1 name: sas-platform-postgres   Now that the manifests are updated you can build and apply using your chosen deployment method.   Now we will validate that the Viya Environment points at the Azure flexible PostgreSQL server by checking the database registration in the SAS Configuration server. The serviceAddress in the output should match the name of your target PostgreSQL database, for example: sasgnn-p21077-default-flexpsql.postgres.database.azure.com   WARNING: Do not continue to the next step until you have validated that the Consul registration has switched to external PostgreSQL.   HTTP_PROTOCOL=https CONSUL_SVC_NAME=sas-postgres kubectl exec -it sas-consul-server-0 -c sas-consul-server -- bash -c "export CONSUL_HTTP_ADDR=$HTTP_PROTOCOL://localhost:8500; /opt/sas/viya/home/bin/sas-bootstrap-config catalog service $CONSUL_SVC_NAME"   Expected output:   { "items": [ { "address": "sasgnn-p02189-default-flexpsql.postgres.database.azure.com", "createIndex": 26687, "datacenter": "viya", "modifyIndex": 26687, "node": "sas-postgres", "nodeMeta": null, "serviceAddress": "sasgnn-p21077-default-flexpsql.postgres.database.azure.com", "serviceEnableTagOverride": false, "serviceID": "sas-postgres", "serviceMeta": {}, "serviceName": "sas-postgres", "servicePort": 5432, "serviceTags": [ "postgres", "public", "data-service-postgres", "ssl" ], "serviceWeights": { "passing": 1, "warning": 1 }, "taggedAddresses": null } ] }   If the serviceAddress still points to the internal instance or there are any other issues you can review the data server operator custom resource and logs.  There should be no errors in the data server operator logs and the operator definition should point to the new azure PostgreSQL flexible server.  If you find errors check your overlays and re-build and apply.   kubectl describe dataservers.webinfdsvr.sas.com kubectl logs -l app=sas-data-server-operator   To complete the process, restart the SAS Viya Platform by running the stop and then the start job and wait for SAS Viya to start.   kubectl create job sas-stop-all-`date +%s` --from cronjobs/sas-stop-all RUNNING_PODS=$(kubectl get pods --field-selector=status.phase=Running -o go-template='{{len .items}}') until [ $RUNNING_PODS -lt 2 ]; do \ sleep 5; echo Running Pods: $RUNNING_PODS; \ RUNNING_PODS=$(kubectl get pods --field-selector=status.phase=Running -o go-template='{{len .items}}'); \ done kubectl create job sas-start-all-`date +%s` --from cronjobs/sas-start-all kubectl wait --for=condition=ready pod --selector='app.kubernetes.io/name=sas-readiness' --timeout=2700s   If you are satisfied that the data has been transferred successfully and SAS Viya is functioning, then you can stop the internal PostgreSQL instance. Shut down the internal PostgreSQL.   kubectl patch postgresclusters sas-crunchy-platform-postgres --type json --patch '[{"op":"replace", "path": "/spec/shutdown", "value": true}]'   Expected output:   postgrescluster.postgres-operator.crunchydata.com/sas-crunchy-platform-postgres patched   Validate   Before removing the internal PostgreSQL instance, it is important to validate that Viya is functioning and that the content and configuration has been transferred.  In this step we use the ValidateViya tool to execute some validation tests.    python3 /opt/pyviyatools/validateviya.py -c /mnt/workshop_files/initenv/validation/tests.json -d /tmp/testresults -o report-full   Review the HTML Report Output in /tmp/testresults. The reports should show the folders where content was loaded (gelcorp and gelcontent) and the caslibs are all available.     Delete internal PostgreSQL.   Now we can remove the internal PostgreSQL. A script is provided to delete internal PostgreSQL servers and its Persistent Volume Claims(PVCs). Answer yes when prompted.   chmod 755 ~/project/deploy/pgtransfer/scripts/sas-postgres-delete.sh ~/project/deploy/pgtransfer/scripts/sas-postgres-delete.sh --namespace gelcorp   Finally, you can remove all references to the internal PostgreSQL from the kustomization.yaml.   ansible localhost -m lineinfile -a "path=~/project/deploy/${GELLOW_NAMESPACE}/kustomization.yaml regexp='crunchy' state='absent'"   If all the Viya environments in your cluster have been switched to using an external PostgreSQL you can delete the Crunchy data cluster-wide resources.   kubectl delete crd pgupgrades.postgres-operator.crunchydata.com kubectl delete crd postgresclusters.postgres-operator.crunchydata.com   Wrap Up   The PostgreSQL Data Transfer Tool for the SAS Viya Platform allows you to move content from an internal instance of PostgreSQL to an external instance. The transfer tool only copies the data in the internal PostgreSQL instance to an external PostgreSQL instance. Required post-transfer steps change the connection information so that the existing SAS Viya platform deployment uses the external instance instead of the internal one. Later steps remove the internal instance. In this post we have walked through the process of switching from an internal PostgreSQL instance to an external instance using Azure Flexible PostgreSQL server.     Find more articles from SAS Global Enablement and Learning here. ... View more

SAS Viya backup and restore has recently introduced some exciting new features that offer greater flexibility in backing up and restoring SAS Viya Environments. In this post, I will provide an overview of these features, and discuss the advantages and disadvantages of each approach. This will help you determine which backup and restore strategy best suits your requirements.   As a reminder, the tools are commonly used in three main scenarios: restoring in place, SAS Viya 4 to 4 migration, and disaster recovery. Let’s start by taking a look at the classic backup and restore functionality that we have had since SAS Viya 4 was released.   SAS Viya Application Backup and Restore   Backup with SAS Viya application backup is the default. This is the functionality we have had available since the release of SAS Viya.   A backup includes a complete backup of PostgreSQL databases, Configuration Server, and CAS metadata and data. The PostgreSQL backup and restore are performed using pg_dump and pg_restore. These utilities make a complete copy of the database into and out of the backup package. CAS data and metadata are likewise copied in full to and from the package.   Select any image to see a larger version. Mobile users: To view the images, select the "Full" version at the bottom of the page.   Backups can be used for disaster recovery, SAS Viya 4 to 4 migration, and in-place restores. While this method is versatile and can be used in all scenarios, it may run more slowly when working with large amounts of PostgreSQL data. Similarly, the CAS backup will be slower when the default caslibs contain large volumes of data.   Backup and Restore Using Provider-Specific Processes for PostgreSQL in addition to SAS Viya Application Backup   You can exclude the PostgreSQL data from the backup process and use the PostgreSQL provider to manage that part of the backup. To do this set the environment variable INCLUDE_POSTGRES=False in the backup configMap. A full backup excluding PostgreSQL includes a complete backup of the Configuration server, CAS metadata and data, but does not include any of the PostgreSQL databases.   The PostgreSQL database backup is managed by the PostgreSQL provider. No PostgreSQL data is written to the backup package. For more details on this approach check out this article which covers its usage on Azure using Azure Flexible PostgreSQL server. This approach can also be used in conjunction with the internal PostgreSQL server using the pgBackRest utility of Crunchy PostgreSQL.     The PostgreSQL provider must support Point-in-time restore (PITR). In addition, the retention period of the PostgreSQL backup and the SAS Viya backup should be synchronized to ensure both parts of the backup are available at the point in time selected for restoration. All backup and restore scenarios are supported, except in cases where internal Crunchy PostgreSQL is used; in these cases, only in-place restore is supported.   Both the backup and the restore process are significantly faster when using this approach. The backup PVC’s will require less storage space since no PostGresSQL data is included.   Backup and Restore Using Provider-Specific Processes for PostgreSQL and Incremental SAS Viya Application Backup   A recent addition to backup and restore functionality added the ability to incrementally backup CAS data. An incremental backup strategy involves storing only the data that has been modified since the last backup was taken. The first backup in the series is a complete backup, and the subsequent backups only store the modified or new data, which is the incremental differences. This approach helps to save time in performing frequent backups.   There are two parts to the CAS backup: the CAS metadata (permstore) and the CAS data in the default caslibs. This approach creates a full backup of CAS metadata but an incremental backup of CAS data.   To use this approach, you must: Enable the Viya incremental backup job. Use Provider-Specific Processes for PostgreSQL by specifying INCLUDE_POSTGRES=false The incremental backup job, when enabled, runs every day (except Sunday) at 6 am. The retention period of the PostgreSQL backup and the SAS Viya backup should be synchronized. In addition, to perform incremental backups, a periodic full SAS Viya backup is required.   Incremental backups are supported for all scenarios except when internal Crunchy PostgreSQL is used. Again, in the latter case only in-place restore is supported.  When using this approach there are significant improvements in the speed of PostgreSQL and SAS Viya backups, especially if you have a lot of data in the databases and/or the default caslibs. In addition, the volume of data stored in the backup PVC’s will be a lot smaller.   Comparison of Backup and Restore Approaches   The table below summarizes the 3 approaches.       The recent changes mean that for all scenarios your backup package looks a little different. The format of the backup ID has changed. The backup ID is a date timestamp value that uniquely identifies each backup.   The SAS-BACKUP-ID of the full Viya backup is in the format of YYYYMMDD-HHMMSSF (the F as it signifies that this is a full backup). For example, 20240228-160410F   The SAS-BACKUP-ID of an incremental backup is in the format YYYYMMDD-HHMMSSF_YYYYMMDD-HHMMSSI. For example, 20240228-160410F_20240301-120310I. The incremental backup ID's first timestamp refers to the last full backup, and the second to the last incremental backup.     In both cases, YYYY represents the year, MM represents the month, and DD represents the day of the month. HH represents the hour, MM represents the minute, and SS represents the second. All times are in UTC.   For SAS Viya 4 to 4 migration and disaster recovery, you must copy all backup packages, including one full backup and all related incremental backups, from the source to the target.   Wrap Up   SAS Viya has recently been updated to improve its backup and restore functionality, providing more flexibility in how the process is managed. With the new changes, it’s now possible to perform point-in-time restores of PostgreSQL and incremental backups of the CAS filesystem. This allows for more efficient use of resources and faster processing of backup and restore. If you’re not concerned about time and resource usage, the existing backup and restore process remains a good option that’s easy to use.   I would like to thank Dawn Schrader, Penny Downey, JK Min and Scott McCauley for their valuable input to this post.   Please check out the Backup and Restore documentation for more details:   Backup and Recovery: Workflows Perform a Restore Using Only SAS Viya Processes for All Content Perform a Restore Using Provider-Specific Processes for PostgreSQL Schedule an Incremental Backup Perform an Incremental Ad Hoc Backup Restore PostgreSQL Cluster from pgBackRest Backup Completely or Point-In-Time-Recovery (PITR)   Check these related blog posts:   Restore a Viya Backup with SAS Viya Backup and Database Point in Time Restore | Global Enablement & Learning Blog SAS Viya: Restore a Backup in 2023.10 and later. | Global Enablement & Learning Blog A first look and Backup and Restore in Viya 4 | Global Enablement & Learning Blog (sas.com)   Find more articles from SAS Global Enablement and Learning here. ... View more

Traditionally SAS Viya has relied on internal processes to perform backup and restore. A recent change has added support for using third-party functionality to backup the SAS Infrastructure Data Server portion of the Viya deployment. In this post I will introduce this new feature and demonstrate how you can use it, using a SAS Viya deployment on Azure Kubernetes Service (AKS).   It is important to note that this is an addition of functionality, not a replacement. Instead of a single workflow that uses SAS Viya application backup tools to perform a backup and restore, there are now two workflows available:   SAS Viya Application Backup Only SAS Viya Application Backup and Cloud-Provider for external PostgreSQL ​   To use the second workflow the PostgreSQL provider must support Point-In-Time-Recovery (PITR) of the external PostgreSQL server to a custom restore point based on date and time. Many cloud-provider databases do provide this functionality. For example, the Azure Doc states:   “Azure Database for PostgreSQL - Flexible Server automatically performs regular backups of your server. You can then do a point-in-time recovery (PITR) within a retention period that you specify.”   Check out this page for more details on Azure Database for PostgreSQL - Flexible Server   The benefit of using PITR is that it can be significantly faster to perform a restore than the traditional SAS Viya Backup which uses pg_dump and pg_restore The traditional approach can be very slow when there is a large volume of data in the PostgreSQL database.   Let's look at how to backup and restore using SAS Viya application backup and Azure Database for PostgreSQL - Flexible Server PITR.   Backup   To rely on the Azure Flexible PostgreSQL server point-in-time restore functionality for the SAS Infrastructure Server backup we will set a parameter that will exclude the Infrastructure Data Server from the SAS Viya Application backup. To exclude the SAS Infrastructure Data Server from the backup add the parameter INCLUDE_POSTGRES='false' to the sas-backup-job-parameters configmap. Add the following code to the configMapGenerator section of kustomization.yaml and perform a build and apply.   configMapGenerator: [... previous resource items ...] - name: sas-backup-job-parameters behavior: merge literals: - INCLUDE_POSTGRES="false" [...]   With the backup configMap updated, we can now run an ad-hoc backup. In this step, we start the backup job from the scheduled backup cronjob and give it the name sas-scheduled-backup-job-adhoc-001.   kubectl create job --from=cronjob/sas-scheduled-backup-job sas-scheduled-backup-job-adhoc-001   Follow the log of the backup job until it is completed.   kubectl logs -f job/sas-scheduled-backup-job-adhoc-001 -c sas-backup-job | gel_log   Check the job log to make sure the backup ran successfully.    Select any image to see a larger version. Mobile users: To view the images, select the "Full" version at the bottom of the page.   NOTE: in the previous command we pipe to a custom function(gel_log) that uses JQ to reformat the log from JSON to a more human-readable format.   If we review the backup package in sas-common-backup-data PVC there will be no /sas-common-backup-data/${backupid}/default/postgres directory because the SAS Viya Infrastructure data server PostgreSQL database was not included in the backup. The diagram below shows the overall process and result.     Restore   Prepare for Restore   The first step in the restore is to get the backup ID of the backup to restore. Each backup is identified using a timestamp value called the backup ID. When performing a restore that includes PITR we use the timestamp value to identify :   The SAS Viya backup package AND Restore "point-in-time" within the PosgtGresSQL backup.   In this step, retrieve the backup ID from the job that created the backup.   backupid=$(yq4 eval '(.metadata.labels."sas.com/sas-backup-id")' <(kubectl get job sas-scheduled-backup-job-adhoc-001 -o yaml)) echo Backup Id is: $backupid   Output:   Backup Id is: 20240104-150710   Recently the format of the SAS Viya backup ID changed. The new format of the backup ID is YYYYMMDD-HHMMSSF (the F can be ignored as it signifies that this is a full backup) In YYYYMMDD, YYYY is an integer that represents the year, MM is an integer that represents the month of the year, and DD is an integer that represents the day of the month. In HHMMSS, HH is an integer representing the hour, MM is an integer representing the minute, and SS is an integer representing the second. All time is UTC.   NOTE: the new backup ID format is backward compatible with the old format.   The restore point format for Azure Postgres PITR is YYYY-MM-DDTHH:MM: SSZ eg. "2021-05-05T13:10:00Z". You can use this command to convert the backup ID to the format of the restore point.   restorePoint=${backupid:0:4}-${backupid:4:2}-${backupid:6:2}T${backupid:9:2}:${backupid:11:2}:${backupid:13:2}Z:${backupid:12:2}Z echo Restore Point is: $restorePoint   Output:   Restore Point is: 2024-01-04T15:07:10Z:71Z   An important feature of PITR is that the process does not restore to the same PostgreSQL server instance. To restore you need to provide a new instance name, which should be unique. It's a good practice to have a naming convention for your instances as you may end up with multiple instances in a cluster. For our purposes, we will just prepend the backup ID to the existing Flexible PostgreSQL server name eg. ${backupid}-blitz-0331-rg-default-flexpsql. In this step, we create an environment variable with the new name.   RESTORE_DB_NAME=d${backupid,,}-${GELENABLE_PREFIX}-default-flexpsql echo Restored DB instance: ${RESTORE_DB_NAME}   Output:   Restored DB instance: d20240104-150710f-boxer-p02095-default-flexpsql   Perform Point-in-Time Restore   The restore can be performed in the Azure Portal or using the Azure CLI. In the Azure Portal select the Resource Group of your Viya Deployment. Under the listed resources, select the Azure PostGresSQL Flexible server. Within the flexible server under Settings on the right, select Backup and Restore. Follow these steps to restore:   Step 1. Select the Fast Restore option for the Automated Backup with the Completion timestamp before, but closest to the restore point.     Step 2. Select the radio button Select a custom restore point and provide.   The date and time of the restore point that matches the backup ID. Give the new database instance a unique name.     Select Review and Create and then Create.   The deployment of the new instance will start with the message "Deployment in Progress". When it is completed the UI will output the message "Your deployment is complete".  This will complete the restoration of the Viya Infrastructure data server. It is restored to the new instance name and in Azure you now have two instances, the Viya environment still points to the original instance. In the following steps of the restore, we will switch Viya to point to the restored instance.   Restore the SAS Viya Backup   When using PITR for the PostGresSQL database you must select a backup package that does not include the SAS Viya Infrastructure Data Server data. The restore process happens in two steps. These steps are summarized here and covered in more detail in my previous post.   Step 1 The Restore Job   Restores the SAS Configuration Server Stop the CAS Server(s)   Step 2 Clear the CAS PV's and restart CAS in RESTORE mode   Delete existing data from the CAS PV's Restores CAS data and content   Run the Restore Job   Firstly, identify the sas-restore-job-parameters configMap that needs to be modified. This step returns the config map for the restore job.   restore_config_map=$(kubectl describe cronjob sas-restore-job | grep -i sas-restore-job-parameters | awk '{print $1}'|head -n 1) echo The current restore Config Map is: $restore_config_map   Output:   The current restore Config Map is: sas-restore-job-parameters-hgd4ftbmmm   Edit the configMap to set the restore parameters.   SAS_BACKUP_ID to the backup ID of the package to restore. SAS_DEPLOYMENT_START_MODE to RESTORE. AUTO_SWITCH_POSTGRES_HOST to True. DATASERVER_HOST_MAP to map the name of the restored PostGresSQL Server.   We have stored these values in environment variables in prior steps. This code updates the restore configMap with the relevent values and checks the values have been updated.   RESTORE_DB_HOST=${RESTORE_DB_NAME}.postgres.database.azure.com kubectl patch cm $restore_config_map --type json -p '[ {"op": "replace", "path": "/data/SAS_BACKUP_ID", "value":"'${backupid}'"},{"op": "replace", "path": "/data/SAS_LOG_LEVEL", "value":"DEBUG"},{"op": "replace", "path": "/data/SAS_DEPLOYMENT_START_MODE", "value":"RESTORE"},{"op": "replace", "path": "/data/AUTO_SWITCH_POSTGRES_HOST", "value":"True" },{"op": "replace", "path": "/data/DATASERVER_HOST_MAP", "value":"sas-platform-postgres:'${RESTORE_DB_HOST}'"}]' kubectl describe cm $restore_config_map     Start the Restore Job from the Restore cronjob. This process will restore the SAS Configuration Server. In addition, it will stop the CAS server to prepare for restore of the CAS server.   kubectl create job --from=cronjob/sas-restore-job sas-restore-job   After giving the job some time to run you can check for specific messages in the log to check the status.   kubectl logs -l "job-name=sas-restore-job" -c sas-restore-job --tail 1000 | klog | grep "restore job completed successfully" -B 3 -A 10   IMAGE   Restore the CAS Server   With the restore job completed the second step will restore the CAS server. This process is covered at a high level here but is covered in more detail in this post.   The restore job has stopped all CAS Servers in the environment. To restore CAS, the CAS Server will be started in RESTORE mode and data and configuration will be restored during server startup. Firstly, let’s check that CAS is not running.   kubectl get pods --selector="app.kubernetes.io/managed-by==sas-cas-operator"   Output:   No Resources in yournamespace namespace.   The process uses two provided CAS scripts.   sas-backup-pv-copy-cleanup.sh deletes the existing data from the CAS PV's. scale-up-cas.sh starts the CAS server(s) in RESTORE mode.   Make the provided CAS scripts executable.       Restoring the data to the two CAS file PVCs requires a clean volume. Run the sas-backup-pv-copy-cleanup.sh script to clean up the CAS PVCs. This step deletes the existing data on the CAS permstore(cas-default-permstore) and CAS data(cas-default-data) PVCs.   ~/project/deploy/${current_namespace}/sas-bases/examples/restore/scripts/sas-backup-pv-copy-cleanup.sh gelcorp remove "default"   Startup the CAS Server   ~/project/deploy/${current_namespace}/sas-bases/examples/restore/scripts/scale-up-cas.sh gelcorp "default"   When the CAS server starts it checks the Restore Job configMap attribute SAS_DEPLOYMENT_START_MODE. If it is set to RESTORE, the CAS server will start and restore the data from the directory that matches the BACKUP_ID in the sas-cas-backup-data PVC.   Check the log to see if the CAS server performed the restore. The logs should show the start of the restore process that restores the backup content to the target CAS persistent volumes.   kubectl logs sas-cas-server-default-controller -c sas-cas-server | gel_log | grep -A 10 "RESTORE"   Finalize the Restore   To complete the switch over to the newly restored Postgres instance, restart all the services with the sas-stop-all and sas-start-all cronjobs. This code runs the stop job and waits for all the PODS on the namespace to stop before running the start and job and then waiting for Viya to start.   kubectl create job --from cronjobs/sas-stop-all restart-stop RUNNING_PODS=$(kubectl get pods --field-selector=status.phase=Running -o go-template='{{len .items}}') until [ $RUNNING_PODS -lt 2 ]; do echo Running Pods: $RUNNING_PODS; RUNNING_PODS=$(kubectl get pods --field-selector=status.phase=Running -o go-template='{{len .items}}'); done kubectl create job --from cronjobs/sas-start-all restart-start   To ensure that SAS Viya 4 is referencing the restored PostgreSQL instance, execute the following command for each PostgresSQL instance in the cluster. The command should show the address and services as the new name that you gave to the PostgresSQL database.   HTTP_PROTOCOL=https CONSUL_SVC_NAME=sas-postgres kubectl exec -it sas-consul-server-0 -c sas-consul-server -- bash -c "export CONSUL_HTTP_ADDR=$HTTP_PROTOCOL://localhost:8500; /opt/sas/viya/home/bin/sas-bootstrap-config catalog service $CONSUL_SVC_NAME" { "items": [ { "address": "d20240115-134651f-blitz-p02095-default-flexpsql.postgres.database.azure.com", "createIndex": 692506, "datacenter": "viya", "modifyIndex": 692506, "node": "sas-postgres", "nodeMeta": null, "serviceAddress": "d0240115-134651f-blitz-p02095-default-flexpsql.postgres.database.azure.com", "serviceEnableTagOverride": false, ........   Finally, we will reset the SAS restore job configMap parameters.   kubectl patch cm $restore_config_map --type json -p '[{ "op": "remove", "path": "/data/SAS_BACKUP_ID" },{"op": "remove", "path": "/data/SAS_DEPLOYMENT_START_MODE"} ,{"op": "remove", "path": "/data/AUTO_SWITCH_POSTGRES_HOST" },{"op": "remove", "path": "/data/DATASERVER_HOST_MAP"} ]'   The Viya manifests point to the PostgreSQL server used before the restore. An important final step ensures that when any future changes are made Viya continues to point to the restored server.  To ensure that Viya continues to point to the restored server update the manifests to point to the new host for the PostgreSQL server. The file to be updated is usually at $deploy/site-config/postgres /{ POSTGRES-SERVER-NAME }-dataserver-transformer.yaml. Here is an example of an updated file.   apiVersion: builtin kind: PatchTransformer metadata: name: sas-platform-postgres-dataserver-transformer patch: |- - op: replace path: /spec/users/0/credentials/input value: secretRef: name: platform-db-user usernameKey: username # For internal use, do not modify passwordKey: password # For internal use, do not modify - op: replace path: /spec/registrations/0/host value: d20240115-134651f-boxer-p02095-default-flexpsql - op: replace path: /spec/registrations/0/port value: 5432 - op: replace path: /spec/ssl value: true target: group: webinfdsvr.sas.com kind: DataServer version: v1beta1 name: sas-platform-postgres   Considerations   The Azure Flexible server for Postgres has a default retention period of 7 days. The retention period for the Postgres should be changed so that it is in synch with the SAS Viya backups. SAS Viya backups have a retention period of 30 days by default. Matching retention periods will ensure that if we need to restore a backup the required PostgreSQL and SAS data is available.   The restore process created a new PostGresSQL instance. You may need to keep the old and new instances around for a while to provide the ability to do a PITR during the 30-day retention period.   Wrap-Up   The addition of support for using third-party functionality to backup and restore the SAS Viya Infrastructure Data Server portion of the Viya deployment is a welcome update. This support depends on the database supporting Point-in-time restore. This addition adds flexibility to the SAS Viya Backup and Restore functionality and will be particularly useful when the volume of data stored in the Infrastructure Data Server is large. The SAS Viya Administration Backup and Restore documentation has been updated to cover the new workflow.     Find more articles from SAS Global Enablement and Learning here. ... View more

SAS Viya stable 2023.10 introduces a simplified approach to restoring a Viya backup. In prior releases, an administrator was required to build and apply temporary manifests to initiate a restore. In 2023.10 (and forward) this is no longer the case. In this post, I will review the new and improved functionality.   Viya Backup and Restore Review   In a previous post, I provided an overview of SAS Viya 4 Backup and Restore. To summarize, a Viya backup includes:​   ​Content stored in the Infrastructure Data Server​ (reports, folders, rules, etc.) Configuration stored in the SAS Configuration Server​ (application configuration settings etc.) CAS permstore ​(Caslib definitions, access controls, etc.) Default CAS persistent volume content ​(contents of the Public caslib etc.)   Viya backup and restore are implemented using native Kubernetes functionality. Backup and restore are implemented as Kubernetes jobs and cronJobs. The restore process has two high-level steps.   Step 1: The Restore Job restores the SAS Infrastructure Data Server and Configuration Server Step 2: The CAS Server starts in RESTORE mode to restore CAS configuration and data.   What has Changed?   So what has changed? In 2023.10 the restore process performs the same two steps, however, the method of initiating a restore has improved. Until 2023.10 the restore process involved building and applying temporary Kubernetes manifests to perform the two steps; running the Restore Job and restarting CAS in RESTORE mode. After restoring the backup was complete the manifests that manage the deployment had to be reset to their original state and the temporary manifests discarded. The restore process was not particularly easy for the Viya administrator.  The new process does not require the creation of temporary manifests. Let's see how it works now.   Step 1 Running the Restore Job   Each individual backup is identified using a timestamp value called the backup ID. In the command below, we retrieve the backup ID of the backup to restore from the ad-hoc backup job that created the backup.   backupid=$(yq4 eval '(.metadata.labels."sas.com/sas-backup-id")' <(kubectl get job sas-scheduled-backup-job-adhoc-001 -o yaml)) echo ${backupid}   Output:   2023-11-09T15_32_28_628_0700   When selecting a backup to restore you should always check the status.json in the sas-common-backup-data PVC directory to be sure that the backup you are restoring was completed successfully. The status.json contains detailed information about the backup. For a successful backup, the file should have the value: sas.com/sas-backup-job-status: Completed.   The sas-restore-job-parameters configMap is how we pass settings to the restore process.  The configMap will be referenced by the restore job and by the CAS Server when it starts. The two parameters to set for a restore are:   SAS_BACKUP_ID: the backup ID of the backup to restore. For example, 2023-11-09T15_32_28_628_0700 SAS_DEPLOYMENT_START_MODE: RESTORE   The patch command patches the configMap updating the values of SAS_BACKUP_ID and SAS_DEPLOYMENT_START_MODE:   restore_config_map=$(kubectl describe cronjob sas-restore-job | grep -i sas-restore-job-parameters | awk '{print $1}'|head -n 1) echo The current restore Config Map is: $restore_config_map kubectl patch cm $restore_config_map --type json -p '[ {"op": "replace", "path": "/data/SAS_BACKUP_ID", "value":"'${backupid}'"}, {"op": "replace", "path": "/data/SAS_DEPLOYMENT_START_MODE", "value":"RESTORE" }]'   Output:   The current restore Config Map is: sas-restore-job-parameters-bm48bd82bg configmap/sas-restore-job-parameters-bm48bd82bg patched   Using the following command we can view the updated configMap and make sure that the SAS_BACKUP_ID and SAS_DEPLOYMENT_START_MODE parameters are correctly set.   kubectl describe cm $restore_config_map   Select any image to see a larger version. Mobile users: To view the images, select the "Full" version at the bottom of the page.   With the parameters set correctly, we can now start the Restore Job from the Restore cronJob. This process will restore the SAS Infrastructure Data Server and the SAS Configuration Server. In addition, it will stop the CAS server which is a pre-requisite for the second step which will restore the CAS server.   kubectl create job --from=cronjob/sas-restore-job sas-restore-job   Output:   job.batch/sas-restore-job created   You can view the job log as it runs with this command, using the -f parameter to stream the log to the screen.   kubectl logs -l "job-name=sas-restore-job" -f -c sas-restore-job | gel_log     Note: in the previous command we pipe to a custom function(gel_log) to reformat the log from JSON to a more human-readable format. The function is shown below for your information.   gel_log () { jq -R -r '. as $line | try (fromjson| "\(.level | ascii_upcase) \(.timeStamp) '['\(.source)']-' \(.message) " ) catch $line ' }   You can check the status of a restore job with this command. The status should eventually change from "Running" to "Completed".   kubectl get jobs -l "sas.com/backup-job-type=restore" -L "sas.com/sas-backup-id,sas.com/backup-job-type,sas.com/sas-restore-status"   Output:   NAME COMPLETIONS DURATION AGE SAS-BACKUP-ID BACKUP-JOB-TYPE SAS-RESTORE-STATUS sas-restore-job 0/1 42s 42s 2023-11-16T19_48_28_628_0700 restore Running   To make sure the job has run successfully check the log for the message "restore job completed successfully."   kubectl logs -l "job-name=sas-restore-job" -c sas-restore-job --tail 1000 | gel_log | grep "restore job completed successfully" -B 3 -A 1     The restore job will perform a rolling start of many of the SAS Viya Pods. Before moving on to the next step we should check that two of the key PODS (SAS logon and Configuration) are up and running.   kubectl get pods -l app=sas-logon-app kubectl get pods -l app=sas-configuration   Output:   [cloud-user@pdcesx11133 from35]$ kubectl get pods -l app=sas-logon-app NAME READY STATUS RESTARTS AGE sas-logon-app-5dcb9df44d-hb8gd 1/1 Running 0 2m26s [cloud-user@pdcesx11133 from35]$ kubectl get pods -l app=sas-configuration NAME READY STATUS RESTARTS AGE sas-configuration-68b558b8b9-g7dg4 1/1 Running 0 2m46s   Step 2 Restore the CAS Server   With the restore job completed the second step will restore the CAS server. The restore job has stopped all CAS Servers in the environment. To restore CAS, the CAS Server will be started in RESTORE mode and data and configuration will be restored during server startup. Firstly, let's check that CAS is not running.   kubectl get pods --selector="casoperator.sas.com/server==default" -n gelcorp   Expected Output:   No resources found in gelcorp namespace.   To replace the old manifest approach two new scripts are now used to initiate the CAS restore: The scripts are delivered with the deployment assets in the directory sas-bases/examples/restore/scripts. The two scripts are:   sas-backup-pv-copy-cleanup.sh deletes the existing data from the CAS persistent volumes. scale-up-cas.sh starts the CAS server(s).   The scripts are delivered with the deployment assets in the directory sas-bases/examples/restore/scripts. To run the scripts, we need to make them executable.   chmod +x ~/project/deploy/${current_namespace}/sas-bases/examples/restore/scripts/*.sh   The restoration of the data to the two CAS file PVCs requires a clean volume. Run the sas-backup-pv-copy-cleanup.sh script to clean up the CAS PVCs. This step deletes the existing data on the CAS permstore(cas-default-permstore)  and CAS data(cas-default-data) PVCs. The parameters of the script are:   namespace operation tenant list (default for non-multi-tenant)   cd ~/project/deploy/${current_namespace}/sas-bases/examples/restore/scripts/ ./sas-backup-pv-copy-cleanup.sh gelcorp remove "default"   Output:   The cleanup pods are created, and they are in a running state. Ensure that all pods are completed. To check the status of the cleanup pods, run the following command. kubectl -n gelcorp get pods -l sas.com/backup-job-type=sas-backup-pv-copy-cleanup | grep 21bef2c   The script creates a Kubernetes Job that clears the key data from the CAS PVCs so that it can be restored from the backup package. Using the command provided in the script output we can view the status of the job and for more details we can view the log of the job.   kubectl -n gelcorp get pods -l sas.com/backup-job-type=sas-backup-pv-copy-cleanup | grep 21bef2c kubectl -n gelcorp logs -l sas.com/backup-job-type=sas-backup-pv-copy-cleanup     With the CAS PVCs successfully cleaned, we can start up the CAS server(s) using scale-up-cas.sh. The parameters of the script are:   namespace tenant list (default for non-multi-tenant)   cd ~/project/deploy/${current_namespace}/sas-bases/examples/restore/scripts/ ./scale-up-cas.sh gelcorp "default"   casdeployment.viya.sas.com/default patched   When the CAS server starts it checks the Restore Job configMap attribute SAS_DEPLOYMENT_START_MODE.  If it is set to RESTORE, the CAS server will start and restore the data from the directory that matches the BACKUP_ID in the sas-cas-backup-data PVC.   You can check the CAS Server log to see if the restore was performed. The logs will show the start of the restore process and details of the restore of the backup content from the backup package to the target CAS persistent volumes.   kubectl logs sas-cas-server-default-controller -c sas-cas-server | grep -A 10 "RESTORE"   An important final step is to reset all SAS restore job configMap parameters. If we don't perform this step the CAS server will attempt to restore the backup from the package on every restart.   kubectl patch cm $restore_config_map --type json -p '[{ "op": "remove", "path": "/data/SAS_BACKUP_ID" },{"op": "remove", "path": "/data/SAS_DEPLOYMENT_START_MODE"}]'   Wrap Up   Restore of a SAS Viya backup is now initiated using kubectl commands and scripts. The new and improved restore process is currently supported for Backup and Restore and Viya 4 to Viya 4 Migration.  Currently, the old method of building and applying manifests is also supported. There are plans to add support for the new method for Viya 3. x to 4 Migration. I hope you found this useful. In the Backup and Restore area look for more exciting updates and related blog posts in the coming months.   The new restore process is documented:   SAS Viya Platform Administration Backup and Restore: Perform a Restore (2023.11) SAS Viya Platform Administration Migration: Moving from SAS Viya 4: Restore(2023.11)     Find more articles from SAS Global Enablement and Learning here. ... View more

A SAS Model Studio project is a top-level container for analytic work in Model Studio. A Model Studio project contains the data source, the pipelines, and related project metadata (such as project type, project creator, share list, and last update history). SAS Model Studio projects are, by default, saved in Viya Content to the user’s “My Folder”. Content Migration of Viya Model Studio Projects has been troublesome in some cases because of the strict prerequisites. Now we have improved functionality that makes life easier. In this post, I will examine these improvements.   Firstly, what are the prerequisites? To import a Model Studio Project:   The user who owns the project must exist and must have logged on in the target environment​​ Before import, data must be available and loaded into the memory in the target environment (in the same caslib as in the source environment). A user-created pipeline or node template must be promoted separately.   For more details on these items see the documentation here.   These prerequisites, particularly the requirement that the user that owns the project must already exist and have logged in, have at times made it challenging to migrate Model Studio projects.  This requirement makes it difficult to migrate Model Studio Projects when the source owner: is no longer at the organization. doesn’t have authorization to the target. is a different user in the target. is the same in the target, but the user ID is managed differently. For example, on the development server, the id is dev_user1 and on the test server, the id is test_user1.   One solution to the problem is to change the ownership of a Model Studio Project to a user who does exist and has logged on to the target environment. In the past, there was no supported way to change ownership of a project. New functionality in the transfer plugin of the sas-viya CLI now allows an administrator to change the ownership of Model Studio Projects and other content types prior to import.   New functionality in sas-viya transfer   There are two methods to migrate Model Studio projects between environments you can export using: The transfer service using SAS Environment Manager or the sas-viya CLI, this creates a transfer package (JSON file)​ SAS Model Studio creates a ZIP file containing one JSON transfer file per project   The new method of changing ownership works with the transfer method of migrating. The process works like this: Use the transfer plug-ins list-owners command to generate an "owners list" JSON file from the transfer package Modify the generated file to map the old owner to a new owner Use the transfer plug-in replace-owners command and the edited "owners list" file's content to create a new transfer package file with the ownership of the content in the package changed.   NOTE: to be clear the ownership of the object is changed in the transfer package not in the source system.   Changing Ownership of Model Studio Projects for Import to Viya   Let’s look at the process in detail using an example. In this example, Sarah has some Model Studio projects in one Viya environment and she needs to share them with a different user in another environment.   She starts by using SAS Environment Manager to export a transfer package and shares that transfer package with the other user GEL Administrator (geladm).   Review the Transfer Package   Firstly, let’s review the content of the provided transfer package.   packageid=$(/opt/sas/viya/home/bin/sas-viya --output json transfer upload --file /tmp/viyapackages/SarahMyFolder.json | jq | jq -r '.["id"]') echo "View the package with id" $packageid sas-viya --output text transfer show --details --id $packageid   The output shows that the package contains the folder path to Sarah’s “My Folder” and the content of the folder which consists of the Model Studio Project and supporting resources.       Generate the  "owners list" file   The first step is to generate the "owners list" file from the transfer package. To do that we can use the new “list-owners” command pass it the transfer package file and output the results to an owners list file. The owner's list file is in JSON format and contains a map of the existing owner(sourceUserID) to a potential new owner(targetUSERID). The source and target are initially the same.     sas-viya transfer list-owners --source-file "/mnt/workshop_files/SarahModelStudio/SarahMyFolder.json" --owners-list-file /tmp/theowners.json [ { "sourceUserID": "Sarah", "targetUserID": "Sarah" }, { "sourceUserID": "anonymousUser", "targetUserID": "anonymousUser" } ]   Update the "owners list" file and change ownership   To change the ownership of objects in this package edit the file and change the targetUserID to the user in the target system who you want to own the content.     [ { "sourceUserID": "Sarah", "targetUserID": "geladm" }, { "sourceUserID": "anonymousUser", "targetUserID": "anonymousUser" } ]   Using the updated file create a new transfer package with the ownership changed from the user Sarah to the user geladm. Pass to the transfer replace-owners command: --source-file the source transfer package --owners-list-file the update owners list file --target-file a file to store the updated transfer package   sas-viya transfer replace-owners --source-file /mnt/workshop_files/SarahModelStudio/SarahMyFolder.json --owners-list-file /tmp/theowner-geladm.json --target-file /tmp/geladm-modelstudio.json     Notice the output indicates that the ownership in the package has been changed based on the content of the file. Simple as that. Of course, to truly test if this works we need to import the package.   Import the new transfer package file   In SAS Environment Manager import the package (you could also use the CLI). When we open the package in the Environment Manager import interface notice that the content will be imported to the user geladm’s personal folder. This is the first clue that we have successfully changed the ownership of the content.         After walking through the import wizard the summary screen shows the content, including the Model Studio projects was imported successfully into the My Folder area of the geladm user.     In the Content Area of SAS Environment Manager, we can see the content, originally owned by Sarah in the source environment is now owned by geladm and is located in geladm's "My Folder".         Wrap Up   When migrating Model Studio projects there are specific prerequisites required to ensure a successful migration. One of those requirements is that the user that owns the project must already exist and have logged in to the target system. This has at times made it challenging to migrate Model Studio projects. One solution to the problem is to change the ownership of a Model Studio Project to a user who does exist and has logged on to the target environment. New functionality in the transfer plugin of the sas-viya CLI now allows an administrator to change the ownership of Model Studio Projects and other content types prior to import.   For more information:   Content Migration Considerations for SAS Visual Data Mining and Machine Learning   Machine Learning Users Guide: Importing and Exporting Projects   ... View more