Selective backup and restore of SAS Viya Content

2 Likes

A SAS Viya Administration question that I have heard multiple times recently is whether it is possible to perform a partial backup and restore of a SAS Viya environment. In addition to hearing it from colleagues and customers, I also ran into a case where we needed to implement a partial backup in our internal reporting environment. Given the interest, I thought Communities post. 🙂

The SAS Viya Backup and Restore application employs a comprehensive approach to system backup and restoration. It generates a full backup of your Viya system, enabling you to restore the entire backup in the event of an issue or for disaster recovery and migration purposes. This feature provides assurance that your system is fully safeguarded and can be restored from the backup.

However, it is important to note that the restore process is designed to restore the complete Viya backup package, rather than individual components. Consequently, the restoration process may take time and may necessitate temporary system downtime.

What if you want to protect key Viya content and be able to restore that content selectively? Using SAS Viya's transfer service, you can back up only specific assets rather than the entire system, and in return, you can selectively back up individual or groups of content items without any impact on the running environment.

Why use this method?

Granular control: backup and restore only what you need.
Efficient: backup only content that has changed in a certain time frame.
No downtime: Doesn’t require stopping services or system access.

A limitation of this approach is that the user must keep track of dependencies and ensure that dependent objects, such as data used in reports, are also exported.

How do we do it: sas-viya transfer?

You can export the content to a SAS Viya package to backup specific items like reports, models, flows, jobs etc. The export can be performed interactively, but it is most useful to script the process for backup purposes. When the process is scripted it can be run on a schedule. The transfer plugin of the sas-viya CLI can be used to export content from the command line. You can export a folder and its content or individual content items.

Automating the export process takes a few steps, is dependent on a third-party tool, JQ, and requires identifying the Uniform Resource Identifier(URI) of the content to export. A URI uniquely identifies each folder and content item in SAS Viya. The example below shows the export of a single SAS Viya folder.

Identify the URI (a unique identifier) of the folder
Export the folder by passing the URI to the sas-viya transfer export
Identify the package ID of the package created.
Download from the Infrastructure Data Server the package created as a JSON file.

folderid=$(sas-viya --output json folders show --path /gelcontent/GELCorp/Sales  | jq | jq -r  '.["id"]')
sas-viya --output text transfer export -u /folders/folders/$folderid --name SalesContent$folderid
packageid=$(sas-viya --output json transfer list --name SalesContent$folderid | jq | jq -r '.items[]["id"]')
sas-viya transfer download -f /tmp/viya4packages/SalesContent.json --id $packageid

With this approach, we can script the backup of SAS Viya content to JSON package files and selectively restore content.

How do we do it: pyviyatools?

The pyviyatools are command-line tools that call the SAS Viya REST API from Python. They are available on the SAS GitHub site and include several tools for backing up and restoring SAS Viya content. Let’s look at some tools for partially backing SAS Viya content. Any of these tools can be scheduled to perform a partial backup regularly.

To export the same folder shown in the example above, we can use exportfolder.py. This tool accepts the full folder path as a parameter, removing some of the complexity of exporting a folder with the sas-viya CLI.

exportfolder.py -f "/gelcontent/GELCorp/Sales/Reports" -d /tmp/viya4packages --filename SalesContent.json -q

The package with the ID 2080ab30-12a0-41cc-91f8-b748f5c5c4a9 was created.

/usr/local/bin/sas-viya transfer download --file /tmp/viya4packages/SalesContent.json.json --id 2080ab30-12a0-41cc-91f8-b748f5c5c4a9
NOTE: Viya folder /gelcontent/GELCorp/Sales/Reports exported to json file /tmp/viya4packages/SalesContent.json.json

If you want a backup of all your content, exportfoldertree.py will export the complete Viya folder tree. Each subfolder at the root of the folder tree is exported to its own SAS Viya package.

exportfoldertree.py -d /tmp/allfolders -q

Select any image to see a larger version.
Mobile users: To view the images, select the "Full" version at the bottom of the page.

These two tools allow us to back up folders in the SAS Viya folder tree. Since each folder is exported to its own package file, individual folders can be restored. What if we want to get down to the individaul content level and only backup content that has changed. The Snapshotcontent.py tool allows you to pass a Viya folder. It will recurse through the folder and subfolders and export each content item to a JSON package. The user can filter by date so that only content changed in a specific time period is exported. The following command will export all content under the /gelcontent folder. Each content item will be exported to its own JSON package.

python snapshotcontent.py -d /tmp/snapshot_all -f /gelcontent -q

The JSON packages are named for the full folder path and item name, enabling easy identification of individual items to restore.

Adding the -c parameter allows the user to select only content that has changed in the chosen number of days. This makes it easy to only backup specific content items that have changed. The command below will export any content under the /gelcontent folder that has changed in the last day.

python snapshotcontent.py -d /tmp/snapshot_all -f /gelcontent -q -c 1

Snapshot content can be scheduled to run on a daily basis creating a partial and incremental backup of SAS Viya Content.

NOTE: An earlier tool snapshotreports.py does the same for SAS Visual Analytics reports.

Restoring/Importing

On the restore side we have choices to import/restore any of the content from the JSON packages.

Using SAS Environment Manager to import the content, you can select what content is restored from each package.

There are two steps to import with the SAS-Viya CLI. Firstly, upload the JSON package to SAS Viya, capturing the package ID, then pass the package ID to the import command to import the package.

packageid=$(sas-viya --output json transfer upload --file /tmp/viya4packages/SalesContent.json | jq | jq -r '.["id"]')
sas-viya --output text transfer import --id $packageid

The pyviyatools also has a tool that can help with importing. Importpackages.py will loop through all the packages in a directory and import them to SAS Viya.

importpackages.py -d  /tmp/snapshot_today/ -q

There are few related pyviyatools that can also help with export and import of Viya artifacts. The names explain what they do.

exportcaslibs.py
exportgeoproviders.py
exportcustomgroups.py
exportstudioflowcode.py
importcaslibs.py
importconfguration.py

Wrap Up

In summary, the SAS Viya Backup and Restore application provides a full-system backup approach, however selective backup and restore of SAS Viya content can be achieved using alternative methods. The SAS Viya transfer service and pyviyatools offer granular control, efficiency, and minimal downtime, allowing users to selectively backup and restore individual content items or folders. By leveraging automation and scripting tools like sas-viya CLI and pyviyatools, organizations can implement an efficient, tailored partial backup strategy for SAS Viya environments. This strategy should be implemented to compliment but not replace SAS Viya Backup and Restore.

Additional Resources

Find more articles from SAS Global Enablement and Learning here.