A first look at Migration from SAS Viya 3.x to SAS Viya
The latest release of SAS Viya (2020.1 and later) is the future of SAS, and as a result, making it easy to get to this new release is critical. In this article, I will look at how SAS Viya 3.x users can "migrate" to the new SAS Viya (2020.1 and later). For customers who want to get to the new SAS Viya (2020.1) from Viya 3.x or SAS 9.4 there will be two main approaches.
- Full System Migration to a new SAS Viya environment will be the main method to get to Viya 2020.1 from Viya 3.4 or 3.5.
- Content Migration using the transfer service (Environment Manager or sas-admin/viya transfer) will be the supported method to get to Viya 2020.1 and later from SAS 9.4. Content Migration will be covered in a later post.
In this article, I will look at Full System Migration from Viya 3.x.
Migration to SAS Viya 2020.1 and later is supported from Viya 3.4 or Viya 3.5. The diagram below shows the overall process.
Let's look at a high-level at what happens in each phase.
Plan
In the planning phase you should understand the source environment and plan for what you want to migrate and what the characteristics of your target environment will be.
The sas-admin inventory cli plays a key part in migration to SAS Viya . The inventory cli generates the "Plan and Backup Ansible playbook". The playbook is used, in conjunction with the 3.x inventory.ini file, to scan the source deployment. In a previous article I discussed running the Inventory CLI manually, the enhancement to generate and use an Ansible playbook removes much of the initial complexity and makes it easy to both scan and package the source environment.
The playbook uses the inventory file to determine what scans to perform on each machine in the environment, it aggregates the resulting data, centralizes the data, loads it to CAS and delivers the Viya inventory report. The report helps in planning the migration. The report will help you:
- Locate and record all Viya content
- Understand what data is in CAS and plan for its migration
- Document resource requirements for the target environment
- Determine readiness
The scan process also generates a set of yaml files that allow the user to customize what data and files are included in the package when the backup is run.
Backup
The Ansible playbook can also be used to create a backup/migration package. The playbook uses the backup cli to package the Viya system content and configuration. Including:
- SAS Infrastructure Data Server – All instances of Postgres databases
- SAS Configuration Server (Consul) – User configuration stored in Consul
- SAS Message Broker (RabbitMQ) – Configuration only
- SAS Cloud Analytic Services – Access Controls and Caslib Information (CAS permstore)
In addition to the "classic" backup the playbook reads the contentselection yaml files created by the scan, and uses the inventory cli to package content from each tier in the source environment. This process can package:
- Configuration files from the file system e.g. _usermods.sas
- System and user-defined path-based Caslibs and their content e.g. formats, models etc.
- Viya generated resources such astore files and other content
- Optionally other user selected file-system artifacts.
The contentselection yaml files can be edited to customize what data is included in the migration package. SAS recommended values are set but can be modified prior to running the Backup Play.
The migration package is a set of directories on the file system at a location set at the time the playbook is run. It includes two sub-directories, one for the user content and data and another with the classic backup files.
Initial Deployment
A key pre-requisite to doing a migration to SAS Viya is that a out-of-the-box deployment must be created. Changes to the Viya configuration and content should not be made in the initial deployment as the restore process will, in most cases overwrite those changes.
Restore
The restore process happens to a running Viya deployment in a namespace on a Kubernetes cluster. The restore will require elevated privileges in the Kubernetes cluster. The restore user must be able to make:
- changes to the Kubernetes resources (deployments, statefulsets etc.) in the cluster.
- the backup package accessible to nodes in the Kubernetes cluster
- a location for CAS data accessible to CAS in the cluster
The process involves using kubectl and kustomize to modify the kubernetes objects in the cluster to restore the content from the package to the new environment. There are three main steps in the process. The major pre-requisite for each step is that the migration package is made available to the environment via some form of storage.
Step 1 Migrate CAS Configuration
The restore process will create a new CAS Custom Resource definition for SAS Viya based on the CAS settings read from the source environment, and stored in the migration package.
Step 2 Restore the SAS Data Server and SAS Configuration Server (Consul)
The restore job is a Kubernetes job which restores the infrastructure data server content, and consul content from the migration package. During this step Viya target environment pods are scaled to zero with the exception of the Infrastructure Data Server and SAS Configuration Server. In addition the restore job deletes the existing CAS custom resource definition to prepare for the CAS restore.
Step 3 Restore CAS and start the environment
At startup the CAS server detects that it is in migration mode and restores the CAS permstore (CASLIB definitions and authorization settings) and any user created content that was selected to be restored.
As part of the restore:
- SAS Infrastructure Data Server (PostresSQL) content is overwritten.
- Consul properties are partially overwritten and new values are merged in.
- SAS Configuration service properties are overwritten
- CAS perm stores are overwritten.
- Data and file restores are overwritten, and new items are added in.
- SAS Model Manager content is overwritten.
- Analytics projects might be overwritten.
Validation
Validation will be an important step when the migration is completed. An inventory scan can be run in the target environment and the scan results loaded to CAS. The results are used by the SAS Viya Comparison report to compare the source SAS Viya 3.X environment and the target SAS Viya environment. This is a good start invalidation, but as always as part of a migration, you should go through a comprehensive validation of the new environment and its content.
A Final Word
This has been a very high-level introduction to migration from Viya 3.x to Viya. Find more articles from SAS Global Enablement and Learning here.
