Setting Up SAS Viya with GitLab CI/CD Pipelines: Step Three
Let's dive into crafting GitLab CI/CD pipelines tailored for SAS Viya. First, you established a GitLab Kubernetes runner. Next, you crafted a bespoke Docker image suited for your SAS Viya ecosystem and uploaded it to a secure, private GitLab container registry. Read this post to understand what a pipeline for SAS Viya looks like, what the different components are and how to put them all together.
Post Series
In this post series you will learn how to:
- Register the GitLab Kubernetes Runner.
- Create Your Own Docker Image for SAS Viya.
- Use GitLab Pipelines to Execute Commands in SAS Viya, using the Docker image you built as the base for your executor – the current post
Video
Take a look at the following video, in which we demonstrate a GitLab CI/CD pipeline that runs a SAS program in SAS Viya.
GitLab CI/CD Pipeline
Begin by creating a .gitlab-ci.yml file in your repository's root directory:
default:
image: $CI_REGISTRY/$GITLAB_USER_LOGIN/sasgitlabdevops/jass/gitlab-runner-sas:0.07
run-sas-program:
stage: build
script:
- echo "Hello, $GITLAB_USER_LOGIN!"
- echo "CI Registry $CI_REGISTRY"
# test container folders
- ls -la
- ls -la /usr/local/bin
# SAS Viya CLI
- sas-viya plugins list
# TLS Certificate location
- export SSL_CERT_FILE=/usr/local/bin/gelenv_trustedcerts.pem
- echo ${SSL_CERT_FILE}
# SAS Viya CLI profile
- sas-viya --profile dev profile set-endpoint $dev_endpoint
- sas-viya --profile dev profile toggle-color on
- sas-viya --profile dev profile set-output fulljson
- sas-viya --profile dev profile show
- sas-viya --profile dev auth login -user $dev_user -password $dev_passwd
# SAS Viya CLI
- echo Run a SAS program stored in Git
- sas-viya --profile dev batch jobs submit-pgm --pgm-path 1.sas --context default --watch-output --wait-log-list --results-dir ~/
Explanation
Image
default:
image: $CI_REGISTRY/$GITLAB_USER_LOGIN/sasgitlabdevops/jass/gitlab-runner-sas:0.07
The configuration snippet you've provided is designed to pull a specific Docker image from your GitLab Container Registry for use in your CI/CD pipeline. This Docker image is the one you previously constructed for your SAS Viya environment.
Here's a breakdown of the image directive in your .gitlab-ci.yml file:
- `default:` sets the default configuration for all jobs in the pipeline.
- `image:` specifies the Docker image to use.
- $CI_REGISTRY and $GITLAB_USER_LOGIN are placeholders for GitLab's predefined variables;
-
- $CI_REGISTRY will become the URL of the GitLab registry, typically registry.gitlab.com.
- $GITLAB_USER_LOGIN will be replaced with your GitLab username.
The complete image name, including the tag, will be resolved using these variables, ensuring that the correct image is used for the pipeline.
Select any image to see a larger version.
Mobile users: To view the images, select the "Full" version at the bottom of the page.
To learn more about building the Docker image, refer to the associated post.
Executor Pod
The pipeline runs. The Kubernetes runner initiates the execution. The runner creates a temporary Kubernetes executor pod from the pulled Docker image.
Stage, Job and Script
The basic bricks in a GitLab CI/CD pipeline are the stage, which contains jobs. The job contains scripts.
run-sas-program:
stage: build
script:
- echo "Hello, $GITLAB_USER_LOGIN!"
Where:
- run-sas-program job runs inside the build stage:
- script: runs a series of shell scripts: printing variables, listing folders, etc.
SAS Viya CLI commands
- List the plugins.
- Initialize a variable needed by the SAS Viya CLI: the location of the SAS Viya TLS certificates. Recall from step two, we ‘baked in’ the certificate in the Docker image.
- Create a SAS Viya profile using variables declared when we setup the GitLab runner.
- Authenticate against SAS Viya (get an access token).
- Submit a SAS program in batch.
# SAS Viya CLI
- sas-viya plugins list
# TLS Certificate location
- export SSL_CERT_FILE=/usr/local/bin/gelenv_trustedcerts.pem
- echo ${SSL_CERT_FILE}
# SAS Viya CLI profile
- sas-viya --profile dev profile set-endpoint $dev_endpoint
- sas-viya --profile dev profile toggle-color on
- sas-viya --profile dev profile set-output fulljson
- sas-viya --profile dev profile show
- sas-viya --profile dev auth login -user $dev_user -password $dev_passwd
- sas-viya --profile dev batch jobs submit-pgm --pgm-path 1.sas --context default --watch-output --wait-log-list --results-dir ~/
Notice the variables:
- $dev_endpoint is the URL of our SAS Viya environment.
- $dev_user is the SAS user executing the pipeline.
- $dev_passwd is the credential for SAS Viya CLI authentication.
These are Project variables, defined under Settings > CI/CD > Variables in GitLab.
From a security perspective you should mask sensitive variables, so that they can’t be printed by accident in a log.
Finally, by using the SAS Viya CLI, you can submit the SAS program to be executed in SAS Viya.
- sas-viya --profile dev batch jobs submit-pgm --pgm-path 1.sas --context default --watch-output --wait-log-list --results-dir ~/
Ultimately, you should see the table successfully loaded in SAS Viya.
Conclusion
In conclusion, this post has walked you through setting up a GitLab CI/CD pipeline designed for SAS Viya, utilizing the Kubernetes runner established in the first step and a custom Docker image crafted in the second step. Looking ahead, we will delve into additional SAS Viya pipelines and various runner options. Thank you for following along!
Acknowledgements
Many thanks to Jan Kostrubiec @JanKostrubiec , Katarzyna Zuk @KatarzynaŻuk , Lars Arne Skår @larsarne , Neil Griffin
@Neil-Griffin , from whom I learned the most about DevOps with SAS Viya and GitLab.
Thank you for your time reading this post. If you liked the post, give it a thumbs up! Please comment and tell us what you think about having conversations with your data. If you wish to get more information, please write me an email.
Find more articles from SAS Global Enablement and Learning here.
