BookmarkSubscribeRSS Feed

How to Build an Azure Self-Hosted Agent for SAS Viya

Started ‎10-17-2022 by
Modified ‎07-22-2024 by
Views 4,467

Read this post to understand how you can build an Azure DevOps self-hosted agent for SAS Viya using scripts. An agent is a virtual machine. This machine can be hosted in a cloud, in Azure, for example. It can also be hosted on-premises.


Overall Picture

 

The steps to create your own self-hosted agent are:

 

Pre-Requisites

  1. Access to an Azure DevOps organization.
  2. Azure DevOps Organization policies allowing personal access tokens (PAT) use.
  3. Permissions to create an agent pool in the Azure DevOps organization or in a project.


Azure DevOps Organization

You need access to an Azure DevOps organization.


Azure DevOps Organization Policies

Your Azure DevOps organization policies must allow the usage of personal access tokens (PAT). The Azure DevOps user must have sufficient rights to create an agent pool and a self-hosted agent. Managing an Azure DevOps organization may be discussed in a future post.


Create an Azure DevOps Self Hosted Agent

The steps are:

  • Create an Azure DevOps Personal Access Token (PAT).
  • Create an agent pool.
  • Add an agent to that pool.


Create a Personal Access Token

In your Azure DevOps organization, from User settings > Personal access tokens (PAT), create a new PAT.

 

To “recruit” the machine as a self-hosted agent, the machine has to initiate the communication. To communicate securely with your Azure DevOps organization a personal access token is required. The token is only needed for the agent configuration.

 

 

Select the scopes:

  • Custom defined:
    • Agent Pools > Read & manage.
    • Build > Read & execute.


Create an Agent Pool

 

Agent pools can regroup several agents. For example, you can add all the development machines in a pool, the test machines in a second pool and so on. You can also have a pool for cloud machines, another for on-premises machines.

 


Add an Agent in a Pool

In an existing agent pool, add the new machine (New agent). Choose x64 for Linux or CentOS.

 



Jump on the Virtual Machine

The configuration has to be initiated from the VM. SSH to that machine:

cd ~
# adapt the resource group name, the VM name and the SSH key to your environment
export MYUSER=$USER && echo $MYUSER
export JUMPBOXKEY=~/.ssh/gelazuredm-aks-key
JUMPBOXIP=$(az vm list-ip-addresses -g $MYUSER-SCR -n $MYUSER-jump-vm --query "[].virtualMachine.network.publicIpAddresses[0].ipAddress" -o tsv)
echo "jump box Public IP: $JUMPBOXIP"
# SSH connect to your jump VM 
cd ~/.ssh
ssh -i $JUMPBOXKEY jumpuser@$JUMPBOXIP

 

Set Azure DevOps Agent Variables


Scripts are from now on executed on the jump box. To configure the agent, you will need your PAT:

export AZP_TOKEN=paste_here_the_PAT_you_created_in_Az_DevOps
# MYUSER is optional. Only used here as a prefix for other variables.
export MYUSER=your_User_ID # your-6-digit-SAS-userid

 
Set the following environment variables, adapt them to your environment:

export AZP_POOL=$MYUSER-sas-viya-jump-vm # name of your agent pool
export AZP_AGENT_NAME=$MYUSER-azuredm-jump-vm # your agent's name in Azure DevOps agent pool
export AZP_URL=https://dev.azure.com/geldmdevops # URL to your Azure DevOps organization.
export AZP_WORK=_work # working folder for the agent on this VM
export SUDO_USER=jumpuser # user to run the agent configuration

 

Install and Update Packages

You might need to do that before you set your agent version.

sudo apt update
sudo apt -y upgrade
sudo apt install jq -y

 

Set the Agent Version

Set the AZP_AGENT_VERSION environment variable to specify the latest version of the agent.

export AZP_AGENT_VERSION=$(curl -s https://api.github.com/repos/microsoft/azure-pipelines-agent/releases | jq -r '.[0].tag_name' | cut -d "v" -f 2)
echo $AZP_AGENT_VERSION


A YAML pipeline on a Linux machine must be using the latest version of the agent, even if it's pre-release. The agent software is constantly updating, so you can curl the version information from a Microsoft GitHub repo. The command uses jq to read the latest version from the JSON string that's returned.

 

Build the Agent Script

Create a script to install the Azure DevOps agent on this VM:

cd ~
tee ~/build-agent.sh > /dev/null << EOF #!/bin/bash set -e # Select a default agent version if one is not specified if [ -z "$AZP_AGENT_VERSION" ]; then AZP_AGENT_VERSION=2.200.2 fi # Verify Azure Pipelines token is set if [ -z "$AZP_TOKEN" ]; then echo 1>&2 "error: missing AZP_TOKEN environment variable"
  exit 1
fi

# Verify Azure DevOps URL is set
if [ -z "$AZP_URL" ]; then
  echo 1>&2 "error: missing AZP_URL environment variable"
  exit 1
fi

# If a working directory was specified, create that directory
if [ -n "$AZP_WORK" ]; then
  mkdir -p "$AZP_WORK"
fi

# Create the Downloads directory under the user's home directory
if [ -n "$HOME/Downloads" ]; then
  mkdir -p "$HOME/Downloads"
fi

# Download the agent package
curl https://vstsagentpackage.azureedge.net/agent/$AZP_AGENT_VERSION/vsts-agent-linux-x64-$AZP_AGENT_VERSION.tar.gz > $HOME/Downloads/vsts-agent-linux-x64-$AZP_AGENT_VERSION.tar.gz

# Create a working directory to extract the agent package to
mkdir -p $HOME/myagent

# Move to the working directory
cd $HOME/myagent

# Extract the agent package to the working directory
tar zxvf $HOME/Downloads/vsts-agent-linux-x64-$AZP_AGENT_VERSION.tar.gz

# Install the agent software
./bin/installdependencies.sh

# Configure the agent as the sudo (non-root) user
sudo chown $SUDO_USER $HOME/myagent
sudo -u $SUDO_USER ./config.sh --unattended \
  --agent "${AZP_AGENT_NAME:-$(hostname)}" \
  --url "$AZP_URL" \
  --auth PAT \
  --token "$AZP_TOKEN" \
  --pool "${AZP_POOL:-Default}" \
  --work "${AZP_WORK:-_work}" \
  --replace \
  --acceptTeeEula

# Install and start the agent service
sudo $HOME/myagent/svc.sh install $SUDO_USER
sudo $HOME/myagent/svc.sh start
sudo $HOME/myagent/svc.sh status
EOF

ls -la

 
Make the script executable, and then run it.

chmod u+x build-agent.sh
sudo -E ./build-agent.sh


Sudo enables the script to run as the root user. The -E argument preserves the current environment variables, including the ones you set, so that they're available to the script.

 

As the script runs, you can see the VM being transformed in an agent.

Test the Self-Hosted Agent with a Pipeline

 


In your Azure DevOps Organization, select your project, go to pipelines. Edit the pipeline. Copy the pipeline code below and paste it over in your new pipeline:

# Starter pipeline
# Start with a minimal pipeline that you can customize to build and deploy your code.
# Add steps that build, run tests, deploy, and more:
# https://aka.ms/yaml

trigger:
- main

pool:
  name: myuser-sas-viya-jump-vm

steps:
- script: echo Hello, world!
  displayName: 'Run a one-line script'

- script: |
    echo Add other tasks to build, test, and deploy your project.
    echo See https://aka.ms/yaml
  displayName: 'Run a multi-line script'

 

What "tells" Azure Pipelines to run on a self-hosted agent are these lines in the pipeline definition:

 

pool:
  name: myuser-sas-viya-jump-vm

 

The pipeline will pick a suitable, available agent from this agent pool.

 

Supposing your agent pool has more agents, you can constrain the pipeline to run only on a specific agent, with the “demands” clause.

pool:
  name: myuser-sas-viya-jump-vm
  demands:
   - agent.name -equals myuser-jump-vm

 

Conclusion

You configured an Azure virtual machine as a self-hosted agent using scripts. You ran a first Azure pipeline on the self-configured agent.

 

After the build, you can use this virtual machine as an Azure Pipelines self-hosted agent with SAS Viya. This will be the subject of the next post.

Resources Used

Microsoft Learn – Exercise – Create a build agent that runs on Azure.

 

Thank you for your time reading this post. If you liked the post, give it a thumbs up! Please comment and tell us what you think about post content.

 

If you wish to get more information, please write me an email.

 

Version history
Last update:
‎07-22-2024 11:35 PM
Updated by:
Contributors

SAS Innovate 2025: Call for Content

Are you ready for the spotlight? We're accepting content ideas for SAS Innovate 2025 to be held May 6-9 in Orlando, FL. The call is open until September 25. Read more here about why you should contribute and what is in it for you!

Submit your idea!

Free course: Data Literacy Essentials

Data Literacy is for all, even absolute beginners. Jump on board with this free e-learning  and boost your career prospects.

Get Started

Article Tags