Read this post to understand how you can build an Azure DevOps self-hosted agent for SAS Viya using scripts. An agent is a virtual machine. This machine can be hosted in a cloud, in Azure, for example. It can also be hosted on-premises.
The steps to create your own self-hosted agent are:
You need access to an Azure DevOps organization.
Your Azure DevOps organization policies must allow the usage of personal access tokens (PAT). The Azure DevOps user must have sufficient rights to create an agent pool and a self-hosted agent. Managing an Azure DevOps organization may be discussed in a future post.
The steps are:
In your Azure DevOps organization, from User settings > Personal access tokens (PAT), create a new PAT.
To “recruit” the machine as a self-hosted agent, the machine has to initiate the communication. To communicate securely with your Azure DevOps organization a personal access token is required. The token is only needed for the agent configuration.
Select the scopes:
Agent pools can regroup several agents. For example, you can add all the development machines in a pool, the test machines in a second pool and so on. You can also have a pool for cloud machines, another for on-premises machines.
In an existing agent pool, add the new machine (New agent). Choose x64 for Linux or CentOS.
The configuration has to be initiated from the VM. SSH to that machine:
cd ~ # adapt the resource group name, the VM name and the SSH key to your environment export MYUSER=$USER && echo $MYUSER export JUMPBOXKEY=~/.ssh/gelazuredm-aks-key JUMPBOXIP=$(az vm list-ip-addresses -g $MYUSER-SCR -n $MYUSER-jump-vm --query "[].virtualMachine.network.publicIpAddresses[0].ipAddress" -o tsv) echo "jump box Public IP: $JUMPBOXIP" # SSH connect to your jump VM cd ~/.ssh ssh -i $JUMPBOXKEY jumpuser@$JUMPBOXIP
Scripts are from now on executed on the jump box. To configure the agent, you will need your PAT:
export AZP_TOKEN=paste_here_the_PAT_you_created_in_Az_DevOps # MYUSER is optional. Only used here as a prefix for other variables. export MYUSER=your_User_ID # your-6-digit-SAS-userid
Set the following environment variables, adapt them to your environment:
export AZP_POOL=$MYUSER-sas-viya-jump-vm # name of your agent pool export AZP_AGENT_NAME=$MYUSER-azuredm-jump-vm # your agent's name in Azure DevOps agent pool export AZP_URL=https://dev.azure.com/geldmdevops # URL to your Azure DevOps organization. export AZP_WORK=_work # working folder for the agent on this VM export SUDO_USER=jumpuser # user to run the agent configuration
You might need to do that before you set your agent version.
sudo apt update sudo apt -y upgrade sudo apt install jq -y
Set the AZP_AGENT_VERSION
environment variable to specify the latest version of the agent.
export AZP_AGENT_VERSION=$(curl -s https://api.github.com/repos/microsoft/azure-pipelines-agent/releases | jq -r '.[0].tag_name' | cut -d "v" -f 2) echo $AZP_AGENT_VERSION
A YAML pipeline on a Linux machine must be using the latest version of the agent, even if it's pre-release. The agent software is constantly updating, so you can curl the version information from a Microsoft GitHub repo. The command uses jq to read the latest version from the JSON string that's returned.
Create a script to install the Azure DevOps agent on this VM:
cd ~ tee ~/build-agent.sh > /dev/null << EOF #!/bin/bash set -e # Select a default agent version if one is not specified if [ -z "$AZP_AGENT_VERSION" ]; then AZP_AGENT_VERSION=2.200.2 fi # Verify Azure Pipelines token is set if [ -z "$AZP_TOKEN" ]; then echo 1>&2 "error: missing AZP_TOKEN environment variable" exit 1 fi # Verify Azure DevOps URL is set if [ -z "$AZP_URL" ]; then echo 1>&2 "error: missing AZP_URL environment variable" exit 1 fi # If a working directory was specified, create that directory if [ -n "$AZP_WORK" ]; then mkdir -p "$AZP_WORK" fi # Create the Downloads directory under the user's home directory if [ -n "$HOME/Downloads" ]; then mkdir -p "$HOME/Downloads" fi # Download the agent package curl https://vstsagentpackage.azureedge.net/agent/$AZP_AGENT_VERSION/vsts-agent-linux-x64-$AZP_AGENT_VERSION.tar.gz > $HOME/Downloads/vsts-agent-linux-x64-$AZP_AGENT_VERSION.tar.gz # Create a working directory to extract the agent package to mkdir -p $HOME/myagent # Move to the working directory cd $HOME/myagent # Extract the agent package to the working directory tar zxvf $HOME/Downloads/vsts-agent-linux-x64-$AZP_AGENT_VERSION.tar.gz # Install the agent software ./bin/installdependencies.sh # Configure the agent as the sudo (non-root) user sudo chown $SUDO_USER $HOME/myagent sudo -u $SUDO_USER ./config.sh --unattended \ --agent "${AZP_AGENT_NAME:-$(hostname)}" \ --url "$AZP_URL" \ --auth PAT \ --token "$AZP_TOKEN" \ --pool "${AZP_POOL:-Default}" \ --work "${AZP_WORK:-_work}" \ --replace \ --acceptTeeEula # Install and start the agent service sudo $HOME/myagent/svc.sh install $SUDO_USER sudo $HOME/myagent/svc.sh start sudo $HOME/myagent/svc.sh status EOF ls -la
Make the script executable, and then run it.
chmod u+x build-agent.sh sudo -E ./build-agent.sh
Sudo enables the script to run as the root user. The -E argument preserves the current environment variables, including the ones you set, so that they're available to the script.
As the script runs, you can see the VM being transformed in an agent.
In your Azure DevOps Organization, select your project, go to pipelines. Edit the pipeline. Copy the pipeline code below and paste it over in your new pipeline:
# Starter pipeline # Start with a minimal pipeline that you can customize to build and deploy your code. # Add steps that build, run tests, deploy, and more: # https://aka.ms/yaml trigger: - main pool: name: myuser-sas-viya-jump-vm steps: - script: echo Hello, world! displayName: 'Run a one-line script' - script: | echo Add other tasks to build, test, and deploy your project. echo See https://aka.ms/yaml displayName: 'Run a multi-line script'
What "tells" Azure Pipelines to run on a self-hosted agent are these lines in the pipeline definition:
pool: name: myuser-sas-viya-jump-vm
The pipeline will pick a suitable, available agent from this agent pool.
Supposing your agent pool has more agents, you can constrain the pipeline to run only on a specific agent, with the “demands” clause.
pool: name: myuser-sas-viya-jump-vm demands: - agent.name -equals myuser-jump-vm
You configured an Azure virtual machine as a self-hosted agent using scripts. You ran a first Azure pipeline on the self-configured agent.
After the build, you can use this virtual machine as an Azure Pipelines self-hosted agent with SAS Viya. This will be the subject of the next post.
Microsoft Learn – Exercise – Create a build agent that runs on Azure.
Thank you for your time reading this post. If you liked the post, give it a thumbs up! Please comment and tell us what you think about post content.
If you wish to get more information, please write me an email.
Are you ready for the spotlight? We're accepting content ideas for SAS Innovate 2025 to be held May 6-9 in Orlando, FL. The call is open until September 25. Read more here about why you should contribute and what is in it for you!
Data Literacy is for all, even absolute beginners. Jump on board with this free e-learning and boost your career prospects.