SAS Viya DevOps De-Mystified
SAS Communities posts offer lots of amazing content on DevOps for SAS, but all of the technical detail can be a bit confusing. So let's step back and better define what we're talking about exactly.
What "DevOps" Typically Means
While DevOps incorporates many different functions from collaborative development to production monitoring, when most people say "DevOps" what they generally mean is automated code deployment. Instead of developers throwing their finished code "over the wall" to a dedicated team of release engineers who move it through the deployment process, DevOps aims to automate the release of code into production. Undoubtedly, this is an over-simplification. However, it is useful to help people understand what DevOps is all about. For example, while DevOps includes development, no one really uses the term "DevOps" to describe the collaborative development and versioning process. Most people really only say "DevOps" when the operations components come into play.
The "Dev" Part of DevOps, Development
To facilitate versioning and collaborative development, SAS developers use a code repository management and collaboration tool like Git or SVN. Developers create and modify SAS code and other files in their own dedicated repositories and branches dedicated to particular tasks, new features, bug fixes, etc. SAS developers can use SAS Studio's Git integration as well as SAS' integration with Visual Studio Code or their editor of choice to work with Git.
The "Ops" Part of DevOps, Operations
When development is complete, the finished SAS code and other files are pushed to the main branch on the central repository. This action triggers the Ops (Operations) part of DevOps to begin. These "Operations" are carried out by a CI/CD Pipeline. This is a fancy term but all it amounts to is an automated sequence of scripts which perform various administration functions like copying files to different environments. While these scripts may seem complicated, we can keep them in context by remembering that the point of them is to deploy the finished code, i.e. the desired functionality, into production.
The Most Basic SAS Viya DevOps Pipeline
To get a sense of what a DevOps pipeline does, let's start with the simplest possible example and build from there.
The simplest possible CI/CD pipeline is just some command to push the new/modified code to the production environment. In Gitlab, it would look something like this:
That's it, just a Linux SCP command to copy the contents of the shared repository to the production server. This is the essence of DevOps, coordinating development and getting the resulting work into use as quickly as possible.
Beyond Just Promoting Code, Libnames
Promoting code is the heart of the matter but there is generally more configuration and supporting assets that you'll need to make your SAS code work. Let's start with SAS libraries as our first consideration.
We want our code to be exactly the same in our development environment as it is in our production environment. Manually changing code to work in different environments (DEV, TEST, PROD, etc.) can introduce bugs as well as getting us away from our ultimate goal of fully automated code deployment. However, while our code will be the same, our data sources and targets will undoubtedly be different. We don't want to put unit test data in our prod tables, right? Additionally, SAS Viya has rich user interfaces we'll want to support as well as our code.
To support all of Viya, we'll want to configure libraries using the autoexec_code of our environment's server contexts. Again, this sounds complicated but it's just Viya's way of specifying autoexec.sas files in its application data.
In our scheme, we'll actually use an old fashioned autoexec.sas file that developers can put the library definitions straight in and promote up along with the code. We have structured all of our data stores to use environment-identifying strings (DEV, TEST, PROD, ...) somewhere in them so we can use Linux environment variables to specify which environment we are deploying as seen in the snippet below.
In our DevOps Pipeline, we'll set the ENV variable to "PROD," use the envsubst bash utility to resolve the variable, perform some additional formatting, and finally use the Viya command line interface to update the autoexec_code attribute. The resulting pipeline job would look something like this (again in Gitlab).
For a fully functional version, see this GitLab file.
Running SAS Code
Say we need to pre-populate a new table, run automated system tests, or create test data. There is no better tool than SAS. We can find a million reasons to use SAS for DevOps tasks. To run SAS code for DevOps purposes, simply include the code in your repository and execute it using the Viya command line interface.
For a fully functional version, see this GitLab file.
Importing SAS Packages
Git only works with file system files. So, promoting SAS content is not as straight forward as regular SAS code files. We need to export them from DEV and then import them into downstream environments like TEST and PROD. Below is an example of importing a SAS transfer package in a Gitlab CI/CD pipeline.
For More
For more information on DevOps for SAS with a host of facilitators like Gitlab, Jenkins, Azure, etc., check out SAS Communities posts as well as these two courses:
- Using Azure DevOps and Azure Pipelines in SAS® Viya®
- DevOps with SAS® Viya® and GitLab
Find more articles from SAS Global Enablement and Learning here.
