Read the post to understand how you can use Azure Pipelines to install packages, tools and then pass commands from the SAS Viya Command Line Interface (CLI) or pyviyatools. The goal is to interact with SAS Viya hosted in Microsoft Azure from an Azure hosted virtual machine. You can use for this purpose the self-hosted agent you created and registered.
The steps to create your own self-hosted agent are:
The Azure Pipeline code provided further will focus on four things:
The Azure Pipeline code is written in yaml.
The parameters section contains:
MYUSER
: a string used to prefix the SAS Viya URL and Azure resources.sasviyaclitgz
: a compressed archive containing the sas-viya (CLI) executable. This points to the file name sas-viya-cli-1.21.0-linux-amd64.tgz. You can download it from the SAS Downloads Page and upload this file in the Git repository accompanying your Azure pipeline. Your Git repository can be hosted in Azure Repos, GitHub, Bitbucket, Gitlab, just to name a few options.
Select any image to see a larger version.
Mobile users: To view the images, select the "Full" version at the bottom of the page.
The variable defines the self-hosted agent pool name created in How to Build an Azure Self-Hosted Agent for SAS Viya.
You can use parameters in variables. For example, ${{ parameters.MYUSER }}-sas-viya-jump-vm
becomes sbxbot-sas-viya-jump-vm
when MYUSER=sbxbot
.
Azure Pipelines does not allow you to use agent pool names fully stored in a parameter.
# this pipeline configures the SAS Viya self-hosted agent
parameters:
- name: MYUSER
displayName: SAS User ID
type: string
default:
- name: sasviyaclitgz
displayName: SAS Viya CLI TGZ Name
type: string
default: sas-viya-cli-1.21.0-linux-amd64.tgz # Nov 2022
variables:
- name: pool
value: ${{ parameters.MYUSER }}-sas-viya-jump-vm
trigger:
- main
pool:
name: $(pool)
The trigger tells Azure Pipelines to start a build, a new pipeline run, when somebody pushes a change to the main branch of the underlying Git repository.
The pool tells the pipeline to build on (use) a self-hosted agent from your agent pool.
You can structure an Azure pipeline in a hierarchy of stages, jobs, scripts or tasks. The hierarchy helps you keep the pipeline code clear and allows you to build-in dependencies between jobs or stages.
There is only one stage called install
. The installPackages
job clones the Git repository on the agent in the checkout
step. You will notice the checkout step repeated in every job. Further, the next four bash scripts will run on the agent, which is a Linux Ubuntu VM. The scripts will:
jq
(a command-line JSON processor).stages:
- stage: install
displayName: Install
jobs:
- job: installPackages
displayName: Install and Update Needed Packages
steps:
- checkout: self
persistCredentials: true
- script: |
echo 'List directory where Git Repo was cloned: ' && pwd
ls -la
echo Get the folder where the Git repo was cloned
export GITFOLDER=$(pwd)
echo Update Advanced Package Tool, or APT
sudo apt update
sudo apt -y upgrade
displayName: 'Install and Update APT'
- script: |
echo Install Python 3.x with pip3
sudo apt install python3-pip -y
displayName: 'Install Python 3.x'
- script: |
echo Install a Linux Json parser named jq
sudo apt install jq -y
displayName: 'Install JQ'
- script: |
echo Check python
python3 --version
echo Check pip3 is installed
pip3 --version
echo Check jq
jq --version
displayName: 'Check install'
The next job has also four scripts. Their purpose is to:
$clidir
folder.- job: sasviyacli
dependsOn: installPackages
displayName: Install the SAS Viya CLI
steps:
- checkout: self
persistCredentials: true
- script: |
echo 'List directory where Git Repo was cloned: ' && pwd
ls -la
echo Get the folder where the Git repo was cloned
export GITFOLDER=$(pwd)
echo Unpack the tgz
tar zxvf ${{ parameters.sasviyaclitgz }}
# confirm you have the sas-viya executable in your folder
ls -la
displayName: 'Unpack the SAS Viya CLI'
- script: |
echo 'List directory where Git Repo was cloned: ' && pwd
ls -la
echo Get the folder where the Git repo was cloned
export GITFOLDER=$(pwd)
clidir=/opt/sas/viya/home/bin
echo Create $clidir folder
sudo mkdir -p $clidir
echo Move the sas-viya program to its default folder
sudo mv sas-viya $clidir
echo Give exec permissions to sas-viya in $clidir folder
sudo chmod +x $clidir/sas-viya
echo List $clidir folder
ls -la $clidir
displayName: 'Move the SAS Viya CLI'
- script: |
echo Install the SAS-VIYA CLI plugins
clidir=/opt/sas/viya/home/bin
echo Set viya_cli_repository
viya_cli_repository=https://support.sas.com/repos/viyacli/
echo $viya_cli_repository
echo INSTALLING ALL plugins
for p in $($clidir/sas-viya plugins list-repo-plugins --repo SAS | awk 'NR > 3 { print $1 }' )
do
$clidir/sas-viya plugins install --repo SAS ${p#"*"}
done
echo Done installing SAS_VIYA CLI
echo List plugins
$clidir/sas-viya plugins list
displayName: 'Install the SAS Viya CLI'
- script: |
clidir=/opt/sas/viya/home/bin
ls -la $clidir
echo Test the sas-viya cli
$clidir/sas-viya --help
echo List sas-viya cli plugins
$clidir/sas-viya plugins list
displayName: 'Test the SAS Viya CLI'
In addition to scripts, in a job you can have tasks, such as the AzureKeyVault task.
Always use an Azure Key Vault instead of hardcoding passwords or secrets in your pipeline code! Remember that the pipeline code is stored in a Git repository. Even if the repository is private, it may still be consulted by many other users. If the Git repository is made public (by mistake, or forked and made public), well, your passwords will be visible, on the web for anyone to consult. Hence, the need for the key vault.
You should use it to retrieve secrets, such as the $(SASPASS)
password from an Azure Key Vault of your choice.
After the task, the script clones the public GitHub repository that contains the pyviyatools.
Next, it creates a SAS_CLI_PROFILE
to configure the SAS Viya CLI for a particular SAS Viya deployment. Notice how the SSL_CERT_FILE
references the .pem file copied on the self-hosted agent in How to Make Your Virtual Machine Talk with SAS Viya on Azure. Without a valid certificate, any communication attempt will be blocked by SAS Viya. You also have to create certain Network Security Group rules to allow communication with your Azure Kubernetes Service cluster over the port 443 (HTTPS).
Even further, the script logs the user in the SAS Viya CLI, using the $(SASPASS)
password and obtains a SAS Viya access token. Finally, it runs the setup for the pyviyatools. The setup uses the SAS Viya CLI profile created.
- job: pyviyatools
dependsOn: installPackages
displayName: Install the pyViyaTools
steps:
- checkout: self
persistCredentials: true
- task: AzureKeyVault@2
inputs:
azureSubscription: 'GELDM(41baf198-cc93-4165-9882-6804e5fb95c0)'
KeyVaultName: '${{ parameters.MYUSER }}-key-vault'
SecretsFilter: '*'
RunAsPreJob: true
- script: |
echo Clone the repo
cd ~
git clone https://github.com/sassoftware/pyviyatools.git
echo go to the folder
ls ~/pyviyatools
ls
echo Set Variables
echo Initialize needed variables
export SAS_CLI_PROFILE=gelazure
echo $SAS_CLI_PROFILE
export current_namespace=gelazure
export CURL_CA_BUNDLE=~/.certs/${current_namespace}_trustedcerts.pem
echo $CURL_CA_BUNDLE
export SSL_CERT_FILE=~/.certs/${current_namespace}_trustedcerts.pem
echo ${SSL_CERT_FILE}
export REQUESTS_CA_BUNDLE=${SSL_CERT_FILE}
echo ${REQUESTS_CA_BUNDLE}
echo Running with user: ${{ parameters.MYUSER }}
export INGRESS_URL=https://${{ parameters.MYUSER }}-gel.eastus.cloudapp.azure.com
echo Viya URL: ${INGRESS_URL}
clidir=/opt/sas/viya/home/bin
echo SAS-Viya CLI directory is: $clidir
echo Create SAS-Viya CLI Profile
$clidir/sas-viya --profile ${SAS_CLI_PROFILE} profile set-endpoint ${INGRESS_URL}
$clidir/sas-viya --profile ${SAS_CLI_PROFILE} profile toggle-color on
$clidir/sas-viya --profile ${SAS_CLI_PROFILE} profile set-output fulljson
echo Show profile
$clidir/sas-viya --profile ${SAS_CLI_PROFILE} profile show
echo Login in
$clidir/sas-viya --profile ${SAS_CLI_PROFILE} auth login -user sasadm -password $(SASPASS)
echo Show profile after login
$clidir/sas-viya --profile ${SAS_CLI_PROFILE} profile show
echo Install pyviyatools
python3 ~/pyviyatools/setup.py
displayName: 'Install and setup the pyViyaTools'
A pipeline job running on the self-hosted agent functions like a self-contained code unit. That is why every job recreates the SAS_CLI_PROFILE
. The pipeline will also work if you create it once, then you just reuse the profile in other jobs. Personally, I like to keep each job independent. It allows me to copy them in another pipeline or a new job.
The command python3 ~/pyviyatools/loginviauthinfo.py -f ~/.authinfo
obtains a SAS Viya access token using pyviyatools.
- job: testPyviyatools
dependsOn: pyviyatools
displayName: Test the pyViyaTools
steps:
- checkout: self
persistCredentials: true
- script: |
echo Set Variables
echo Create SAS-Viya CLI Profile
echo Initialize needed variables
export SAS_CLI_PROFILE=gelazure
echo $SAS_CLI_PROFILE
export current_namespace=gelazure
export CURL_CA_BUNDLE=~/.certs/${current_namespace}_trustedcerts.pem
echo $CURL_CA_BUNDLE
export SSL_CERT_FILE=~/.certs/${current_namespace}_trustedcerts.pem
echo ${SSL_CERT_FILE}
export REQUESTS_CA_BUNDLE=${SSL_CERT_FILE}
echo ${REQUESTS_CA_BUNDLE}
echo Running with user: ${{ parameters.MYUSER }}
export INGRESS_URL=https://${{ parameters.MYUSER }}-gel.eastus.cloudapp.azure.com
echo Viya URL: ${INGRESS_URL}
clidir=/opt/sas/viya/home/bin
echo SAS-Viya CLI directory is: $clidir
echo Get a SAS Viya Token with pyviyatools
python3 ~/pyviyatools/loginviauthinfo.py -f ~/.authinfo
echo Show setup
python3 ~/pyviyatools/showsetup.py
echo Listing CASLIBS with pyviyatools
python3 ~/pyviyatools/listcaslibs.py
displayName: 'Test Login and Setup with pyViyaTools'
The login requires an .authinfo file, created on the self-hosted agent, before running the pipeline.
cd ~
tee ~/.authinfo > /dev/null << EOF
default user sasadm password ***your_password_here***
EOF
sudo chmod 600 ~/.authinfo
ls -la ~/.authinfo
You will see the global CASLIBS listed, as a proof that your self-hosted agent can now interact with SAS Viya.
You can pass SAS Viya Command Line Interface (CLI) commands from Azure Pipelines or make SAS Viya REST APIs calls from Python, via pyviyatools. The pipelines are using a self-hosted Linux Ubuntu agent from Microsoft Azure.
Thank you for your time reading this post. If you liked the post, give it a thumbs up! Please comment and tell us what you think about post content. If you wish to get more information, please write me an email.
Find more articles from SAS Global Enablement and Learning here.
SAS Innovate 2025 is scheduled for May 6-9 in Orlando, FL. Sign up to be first to learn about the agenda and registration!
Data Literacy is for all, even absolute beginners. Jump on board with this free e-learning and boost your career prospects.