A Pipeline might be the door to the DevOps world. To enter this world, from SAS Viya, you need a key to the door. The key is a self-hosted agent for SAS Viya. A self-hosted agent can be a Docker container. The Docker container agent has several advantages compared to a Virtual Machine. Would you like to know what these advantages are? Or how to create a Docker agent for SAS Viya? Read the post to find out more.
To integrate SAS Viya with Azure DevOps, you will need an agent.
To run a pipeline, your SAS code, and the pipeline definition (YAML) must be stored in the Git repository. On agents, the code from the Git repository is cloned and executed.
The agent interacts with a SAS Viya environment, through:
Azure DevOps Agents:
A Docker agent is really a Docker container that runs somewhere, for example on an Azure VM.
Inspired by Jan Kostrubiec's work with Docker agents for GitLab, I have been experimenting lately with Docker agents for Azure DevOps. I discovered a few advantages while using Docker containers:
The example uses an Azure Linux VM to build your container image.
If you deployed SAS Viya on Azure, the jump-box-vm comes with the SAS Viya deployment. I will be using this machine which is a Linux Ubuntu 20.04.
If you deployed SAS Viya on-premises, pick a Linux machine with inbound and outbound internet connection.
Checklist:
The VM where you will run your container must be able to reach the Load Balancer IP of your SAS Viya deployment. Port 443 must be allowed.
You will need to obtain a PAT, for the communication with the Azure DevOps organization.
The scripts will require several environment variables:
export AZP_TOKEN=Replace_with_the_PAT_you_obtained_previously
export MYUSER=Replace_with_SAS_USER_ID # optional prefix
# after replacement
export AZP_AGENT_NAME=${MYUSER}-dockeragent
export AZP_URL=https://dev.azure.com/My_DevOps_Org
export AZP_POOL=$MYUSER-sas-viya-jump-vm
export AZP_WORK=_work
export SUDO_USER=jumpuser
To build the Docker image you will require a few files:
Create a folder:
mkdir -p ~/dockeragent && cd ~/dockeragent && ls
Upload the .pem file in the above folder. You can obtain the file by running this kubectl command:
export current_namespace=your_SAS_Viya_Namespace
kubectl cp $(kubectl get pod -l app=sas-logon-app -o=jsonpath='{.items[0].metadata.name}'):security/trustedcerts.pem /home/jumpuser/.certs/${current_namespace}_trustedcerts.pem
Download the .tgz file, the SAS Viya CLI from the SAS Downloads Page in the same folder. You can log in with your SAS profile.
Select any image to see a larger version.
Mobile users: To view the images, select the "Full" version at the bottom of the page.
Microsoft proposes a start script in Run a self-hosted agent in Docker. The script performs several actions:
Create a new file. You might need to change the end of line conversion (Notepad++ example: Edit > EOL Conversion > Unix). Copy the contents:
#!/bin/bash
# start.sh
set -e
if [ -z "$AZP_URL" ]; then
echo 1>&2 "error: missing AZP_URL environment variable"
exit 1
fi
if [ -z "$AZP_TOKEN_FILE" ]; then
if [ -z "$AZP_TOKEN" ]; then
echo 1>&2 "error: missing AZP_TOKEN environment variable"
exit 1
fi
AZP_TOKEN_FILE=/azp/.token
echo -n $AZP_TOKEN > "$AZP_TOKEN_FILE"
fi
unset AZP_TOKEN
if [ -n "$AZP_WORK" ]; then
mkdir -p "$AZP_WORK"
fi
export AGENT_ALLOW_RUNASROOT="1"
cleanup() {
if [ -e config.sh ]; then
print_header "Cleanup. Removing Azure Pipelines agent..."
# If the agent has some running jobs, the configuration removal process will fail.
# So, give it some time to finish the job.
while true; do
./config.sh remove --unattended --auth PAT --token $(cat "$AZP_TOKEN_FILE") && break
echo "Retrying in 30 seconds..."
sleep 30
done
fi
}
print_header() {
lightcyan='\033[1;36m'
nocolor='\033[0m'
echo -e "${lightcyan}$1${nocolor}"
}
# Let the agent ignore the token env variables
export VSO_AGENT_IGNORE=AZP_TOKEN,AZP_TOKEN_FILE
print_header "1. Determining matching Azure Pipelines agent..."
AZP_AGENT_PACKAGES=$(curl -LsS \
-u user:$(cat "$AZP_TOKEN_FILE") \
-H 'Accept:application/json;' \
"$AZP_URL/_apis/distributedtask/packages/agent?platform=$TARGETARCH&top=1")
AZP_AGENT_PACKAGE_LATEST_URL=$(echo "$AZP_AGENT_PACKAGES" | jq -r '.value[0].downloadUrl')
if [ -z "$AZP_AGENT_PACKAGE_LATEST_URL" -o "$AZP_AGENT_PACKAGE_LATEST_URL" == "null" ]; then
echo 1>&2 "error: could not determine a matching Azure Pipelines agent"
echo 1>&2 "check that account '$AZP_URL' is correct and the token is valid for that account"
exit 1
fi
print_header "2. Downloading and extracting Azure Pipelines agent..."
curl -LsS $AZP_AGENT_PACKAGE_LATEST_URL | tar -xz & wait $!
source ./env.sh
trap 'cleanup; exit 0' EXIT
trap 'cleanup; exit 130' INT
trap 'cleanup; exit 143' TERM
print_header "3. Configuring Azure Pipelines agent..."
./config.sh --unattended \
--agent "${AZP_AGENT_NAME:-$(hostname)}" \
--url "$AZP_URL" \
--auth PAT \
--token $(cat "$AZP_TOKEN_FILE") \
--pool "${AZP_POOL:-Default}" \
--work "${AZP_WORK:-_work}" \
--replace \
--acceptTeeEula & wait $!
print_header "4. Running Azure Pipelines agent..."
trap 'cleanup; exit 0' EXIT
trap 'cleanup; exit 130' INT
trap 'cleanup; exit 143' TERM
chmod +x ./run.sh
# To be aware of TERM and INT signals call run.sh
# Running it with the --once flag at the end will shut down the agent after the build is executed
./run.sh "$@" & wait $!
Save it as start.sh in the dockeragent folder.
The Dockerfile creates a Docker container (agent) image for a SAS Viya deployment:
# SAS Viya Docker Agent for Azure DevOps
# v 0.2
# Starting Point: https://learn.microsoft.com/en-us/azure/devops/pipelines/agents/docker?view=azure-devops#use-azure-kubernetes-service-cluster
# SAS Viya specific elements
# Tested with SAS Viya Stable and LTS 2023.03
# Added by Bogdan Teleuca 2023 June 23
# Contribution by Jan Kostrubiec
FROM ubuntu:20.04
RUN DEBIAN_FRONTEND=noninteractive apt-get update
RUN DEBIAN_FRONTEND=noninteractive apt-get upgrade -y
RUN DEBIAN_FRONTEND=noninteractive apt-get install -y -qq --no-install-recommends \
apt-transport-https \
apt-utils \
ca-certificates \
curl \
git \
iputils-ping \
jq \
lsb-release \
software-properties-common \
python3-pip # SAS Viya specific elements \
jq # SAS Viya specific elements
RUN curl -sL https://aka.ms/InstallAzureCLIDeb | bash
# SAS Viya specific elements
# Copy the SAS Viya TLS certificate
WORKDIR /home/jumpuser/.certs
COPY ./gelenv_trustedcerts.pem .
# Copy and unpack the sas-viya cli tgz file - Release 2023.06
# Source: SAS Downloads Page https://support.sas.com/downloads/package.htm?pid=2512
WORKDIR /opt/sas/viya/home/bin
ADD ./sas-viya-cli-1.21.10-linux-amd64.tgz .
# Make the SAS-Viya CLI file executable and install all the plugins
RUN chmod +x /opt/sas/viya/home/bin/sas-viya && /opt/sas/viya/home/bin/sas-viya plugins install --repo SAS all --force
# # end specifically for a SAS Viya Agent
# Can be 'linux-x64', 'linux-arm64', 'linux-arm', 'rhel.6-x64'.
ENV TARGETARCH=linux-x64
WORKDIR /azp
COPY ./start.sh .
RUN chmod +x start.sh
ENTRYPOINT [ "./start.sh" ]
Before running the script, the environment variables must be correctly set.
# Check vars
echo $AZP_AGENT_NAME
echo $AZP_URL
echo $AZP_POOL
echo $AZP_WORK
echo $SUDO_USER
# Build the image
docker build -t dockeragent:latest .
# List the image
docker image ls
The build can take 3 to 5 minutes the first time, or even more.
When the build is complete, your Docker image is ready.
Finally, the docker command gets the latest version of the agent, configures it, and runs the agent. It registers the agent in the specified agent pool, in your Azure DevOps Organisation.
docker run -e AZP_URL=${AZP_URL} -e AZP_TOKEN=${AZP_TOKEN} -e AZP_AGENT_NAME=${AZP_AGENT_NAME} -e AZP_POOL=${AZP_POOL} dockeragent:latest
Thanks to my colleague Jan Kostrubiec for inspiring me to try Docker agents and helping me build a decent Dockerfile for SAS Viya.
SAS Viya and Azure DevOps can be easily integrated to realise several interesting use cases for our customers: code versioning, running SAS programs, jobs, SAS Studio flows, calling SAS Viya REST APIs, promoting code and content.
The self-hosted agent is key to using Azure DevOps with SAS Viya.
A Docker container can be a self-hosted agent. The containerised agent has several advantages compared to a Virtual Machine: fast to create and to destroy, small footprint, better use of compute resources, therefore optimizing costs and availability.
Virtual Machine Agents:
SAS Viya and Azure DevOps Use Cases:
Thank you for your time reading this post. If you liked the post, give it a thumbs up! Please comment and tell us what you think about post content. If you wish to get more information, please write me an email.
Find more articles from SAS Global Enablement and Learning here.
Join us for SAS Innovate 2025, our biggest and most exciting global event of the year, in Orlando, FL, from May 6-9. Sign up by March 14 for just $795.
Data Literacy is for all, even absolute beginners. Jump on board with this free e-learning and boost your career prospects.