SAS Viya and Azure Pipelines Microsoft-Hosted Agents

1 Like

The key to working with SAS Viya and Azure Pipelines from Azure DevOps is the agent that you will use. The post explains what an agent is. It then looks at the different types of Azure DevOps agents, Microsoft-hosted and self-hosted. The post focuses on Microsoft-hosted agents, their advantages and disadvantages.

What Are Agents?

Agents are Virtual Machines or containers that run an Azure pipeline. Agents can run on Linux, Windows or macOS Virtual Machines (VM). They can run in a Docker container, as well. The agents are typically grouped in an agent pool. An agent pool is a regrouping of agents that can run pipelines, just like in the example below.

Select any image to see a larger version.
Mobile users: To view the images, select the "Full" version at the bottom of the page.

The communication with Azure DevOps is initiated by the agent. The agents state their availability to your Azure DevOps agent pool. In Azure DevOps we distinguish two types of agents:

Microsoft-hosted.
Self-hosted.

Microsoft-Hosted Agents

The easiest way to use Azure DevOps is with Microsoft-hosted agents. When you run a pipeline, Azure Pipelines does several things:

First, it determines the type of agent that can run your pipeline.
Second, it takes stock of the agents available and acquires an available agent, from the Azure cloud.
Third, it clones the Git repository linked to your pipeline on this agent.
Fourth, it starts running the instructions in the pipeline definition file.

Watch a starter pipeline running on a Microsoft hosted agent.

(view in My Videos)

The Azure pipeline definition is a YAML file. The file indicates that the pipeline must run on a Microsoft hosted agent, an ubuntu VM (pool: vmImage: ubuntu-latest).

# starter pipeline
# Start with a minimal pipeline that you can customize to build and deploy your code.
# Add steps that build, run tests, deploy, and more:
# https://aka.ms/yaml

trigger:
- main

pool:
  vmImage: ubuntu-latest

steps:
- script: echo Hello, world!
  displayName: 'Run a one-line script'

- script: |
    echo Add other tasks to build, test, and deploy your project.
    echo See https://aka.ms/yaml
  displayName: 'Run a multi-line script'

Advantages

Using this type of agent is very convenient.
Microsoft handles the installation, the maintenance and the upgrade.
Certain software is pre-installed: e.g.: Python, AZ CLI, etc.
It is simple and easy to use.
In your Azure DevOps organization, you have X “run” minutes you can consume every month.

Disadvantages

Each time you run a pipeline you get a fresh Virtual Machine from the Azure cloud.
If any software you need is missing, it is up to you to script the install of that software.
The machine is discarded after each run, including the software you had to install!

Can Microsoft-Hosted Agents Be Used with SAS Viya?

In theory, yes. Although, you must consider the following aspects:

Software and Packages

The first reason is related to existing software on the agent. For an Azure pipeline to work with SAS Viya, your agent must pass SAS Viya command-line interface (CLI) commands or SAS Viya REST API requests. To pass SAS Viya CLI commands, you will need to have the SAS Viya CLI installed on the agent. To invoke SAS Viya REST APIs, the agent will require specific Python packages installed. Now, imagine installing the CLIs and the python packages, again and again, every time you want to run something. Obviously, this will add to the execution time.

Networking

A second reason to consider is networking. On Azure, the agent machine must communicate with your SAS Viya Azure Kubernetes Service (AKS) cluster. The two components sit in different virtual networks (vnet). By default, the SAS Viya deployment allows the access only from a very specific IP address range. Your Microsoft-hosted agent will have an IP unknown to the vnet where SAS Viya is running. Any call you will make from this agent will be blocked by the Network Security Group of the vnet. To overcome the issue, you will have to add a new rule in the Network Security Group. This adds more time to the execution time. Also your MS-hosted agent will have a different IP, at the next run.

Authorizations

A third reason is related to authorizations. SAS Viya is using Transport Layer Security TLS. You would need a set of certificates on the agent, for SAS Viya to allow any communication. Your Microsoft hosted agent will not know about these certificates. You will need to copy them, from somewhere secure, on this agent, every time you are running a pipeline. For all these impracticalities, you should use a self-hosted agent. We will discuss self-hosted agents in the next post.

Conclusions

In this post, you read what an Azure Pipelines MS-hosted agent is and its advantages and disadvantages. MS-hosted agents are the go-to option for a variety of tasks. However, for working with SAS Viya, using a self-hosted agent is probably the best option. Self-hosted agents will be discussed in the next post.