This post shows you step by step how to configure a Virtual Machine hosted on Azure to communicate with SAS Viya deployed on Azure. After the configuration, you can use this virtual machine as an Azure Pipelines self-hosted agent. You can also use this machine in a standalone mode, to execute scripts that interact with SAS Viya. The interaction is possible via the SAS Viya Command Line Interface or via SAS Viya REST API requests.
Overall Picture
The post assumes that SAS Viya is deployed on Azure, in an Azure Kubernetes Service (AKS) cluster.
For SAS Viya to work with Azure DevOps and Azure Pipelines, a self-hosted agent is the key piece. A self-hosted agent is a Virtual Machine (VM). The steps to create your own self-hosted agent are:
Azure Network Security Groups and Rules
One of the key objectives, when working with a self-hosted Azure agent, which can be a VM, is to establish a communication line between:
- The virtual network (VNET) where the VM is deployed (left in the figure below).
- The VNET where SAS Viya resources are deployed (right in the figure below).
The communication line is represented by the double yellow – orange arrow in the middle. I deliberately over-simplified the components on the SAS Viya side, to the right.
To understand a Network Security Groups (NSG), let us use an analogy.
A NSG is the security personnel guarding the doors of an event. To attend the event you must be on the attendees list. The event location is the VNET containing the resources you need to access.
Letting packages from our VM enter the location where SAS Viya “performs”, means adding NSG rules.
A rule has:
- A sense: inbound or outbound.
- Is made of source and destination IPs and ports.
- A directive: Allow or Deny.
If your objective is to launch SAS Viya CLI commands, from the VM, to interact with SAS Viya, you should know that the CLI traffic transits typically over port 443.
The next step is to create a NSG rule.
Pre-Requisites: Login, set your subscription and location
You need to run this code from a machine which has the az cli and kubectl installed. You can use the Azure portal Cloud Shell or another Linux machine with cloud access.
# Variables - ADAPT them to your environment!
export MYUSER=$USER
RG=$MYUSER-SCR
LOCATION=eastus
VM=$MYUSER-jump-vm
NSG=$MYUSER-nsg
MYSUB=GELDM
# Actions
# az login # if not logged in already
az account list -o table
az account set --subscription $MYSUB
az config set defaults.location=$LOCATION
SAS Viya Azure Kubernetes Service Network Security Group Rule (Right Side)
The next rule will allow traffic, from the public IP of your VM, JUMPBOXIP, to the SAS Viya Load Balancer Public IP, INBOUND_IP, over port 443. The rule is created in the aks-agentpool-yyyyyyyy-nsg. This NSG is associated to the Azure Kubernetes Service AKS virtual network.
The inbound rule allows outside access, from the VM, through the ingress controller (load balancer), to the SAS Viya pods.
cd ~
# Get the public IP of the Linux Virtual Machine $MYUSER-jump-vm:
JUMPBOXIP=$(az vm list-ip-addresses -g $RG -n $VM --query "[].virtualMachine.network.publicIpAddresses[0].ipAddress" -o tsv)
echo "${MYUSER} Jump Box Vm's Public IP: $JUMPBOXIP"
# find the internal AKS resource group
RESOURCEGRP=${MYUSER}-azuredm-rg
AKS_RG=$(az aks list -g $RESOURCEGRP --query [].nodeResourceGroup -o tsv)
echo "AKS resource group: $AKS_RG"
# find the public IP address of the AKS cluster
INBOUND_IP=$(az network public-ip list --query "[].{tags: tags.type, address: ipAddress}" -o tsv -g $AKS_RG | grep None | cut -f2)
echo "AKS inbound IP: $INBOUND_IP"
# Get the NSG of the VM
echo "VM NSG: $NSG"
echo "Check: $AKS_NSG is filled!"
echo "In the https://portal.azure.com navigate to $AKS_RG and search for the aks-agentpool-yyyyyyyy-nsg"
# Create inbound NSG rule - port 443
az network nsg rule create -g $AKS_RG --nsg-name $AKS_NSG -n Allow443From-${MYUSER}-jump-vm \
--priority 496 \
--source-address-prefixes $JUMPBOXIP \
--source-port-ranges '*' \
--destination-address-prefixes $INBOUND_IP \
--destination-port-ranges '443' --access Allow \
--protocol Tcp --description "Allow access from New JumpBox on 443"
echo "Rule is created in ${AKS_NSG}"
Physically, you will find the aks-agentpool-yyyyyyyy-nsg under the so-called nodeResourceGroup (usually prefixed by MC_ in SAS Viya deployments on Azure).
SAS Viya Certificates
To make your VM “talk” with SAS Viya, you will most likely pass SAS Viya CLI commands or REST API requests.
But before that, you will need to obtain and copy SAS Viya certificate files on the VM. Otherwise any communication attempt with SAS Viya will result in a certificate error.
The process to copy the certificates can be tricky and involves a few steps.
Get the Certificates
### 0. Setup-Environment-Variables
export MYUSER=$USER
echo $MYUSER
### 1. Get the SAS Viya certificates from the AKS Cluster
#### 1.1 Set the KUBECONFIG file. This is the Kubernetes .conf file allowing you to pass kubectl commands with your SAS Viya deployment.
#### On this machine, the file sits in ~/.kube/$MYUSER-aks-kubeconfig.conf
cd ~
export KUBECONFIG=~/.kube/$MYUSER-aks-kubeconfig.conf # adapt to your environment
echo $KUBECONFIG
#### 1.2 Set the current namespace, where SAS Viya is deployed and the context
export current_namespace=gelazure # adapt to your environment
kubectl config set-context --current --namespace ${current_namespace}
kubectl config get-contexts
#### 1.3 Get the kube .conf file
az aks get-credentials --resource-group $MYUSER-azuredm-rg --name $MYUSER-azuredm-aks --admin -y
ls -la ~/.kube
#### 1.4 List the SAS Viya Certificates
cd ~
ls -la ~/.certs
#### You might not have the folder on your machine. Create the folder.
mkdir -p ~/.certs
#### 1.5 Get the SAS Viya Certificates .PEM File
cd ~
kubectl cp $(kubectl get pod -l app=sas-logon-app -o=jsonpath='{.items[0].metadata.name}'):security/trustedcerts.pem ~/.certs/${current_namespace}_trustedcerts.pem
ls -la ~/.certs
#### 1.6 Backup the .PEM File before the copy
cd ~/.certs
echo Change permission before overwriting
chmod 644 gelazure_trustedcerts.pem
chmod 644 gelazure_trustedcerts.bak
ls -la & pwd
echo Backup the certificates before refresh
cp gelazure_trustedcerts.pem gelazure_trustedcerts.bak
ls -la
#### 1.7 Get the SAS Viya Certificates / Refresh .PEM File
cd ~
kubectl cp $(kubectl get pod -l app=sas-logon-app -o=jsonpath='{.items[0].metadata.name}'):security/trustedcerts.pem ~/.certs/${current_namespace}_trustedcerts.pem
ls -la ~/.certs
Copy the SAS Viya Certificates to your Virtual Machine
### 2. Copy SAS Viya Certificates to VM
#### 2.1 Initialize variables. The VM and the resource group name.
echo $MYUSER
echo Assume VM name is $MYUSER-jump-vm # adapt to your environment
echo Get the VM Public IP
JUMPBOXIP=$(az vm list-ip-addresses -g $MYUSER-SCR -n $MYUSER-jump-vm --query "[].virtualMachine.network.publicIpAddresses[0].ipAddress" -o tsv)
echo "JumpBox Public IP: $JUMPBOXIP"
#### 2.2 Initialize variables: CURL CA Bundle and SSH Private Key. You will SSH the file to the VM.
export CURL_CA_BUNDLE=~/.certs/${current_namespace}_trustedcerts.pem
echo $CURL_CA_BUNDLE
export private_key=~/.ssh/gelazuredm-aks-key
cert=${current_namespace}_trustedcerts.pem
echo $cert
#### 2.3 Create the .certs folder on jump box VM. Assign permissions. Copy files. Verify copy:
cd ~/.certs
ssh -o "StrictHostKeyChecking=no" -i ${private_key} jumpuser@${JUMPBOXIP} "sudo mkdir -p ~/.certs"
wait
echo Transfer folder ownership from root to jumpuser:
ssh -o "StrictHostKeyChecking=no" -i ${private_key} jumpuser@${JUMPBOXIP} "sudo chown jumpuser ~/.certs"
wait
echo Set folder permissions on the jump box, just in case:
ssh -o "StrictHostKeyChecking=no" -i ${private_key} jumpuser@${JUMPBOXIP} "sudo chmod -R 755 ~/.certs"
wait
echo Copy the .pem file:
scp -o "StrictHostKeyChecking=no" -i ${private_key} ${cert} jumpuser@${JUMPBOXIP}:~/.certs
wait
echo Finally, list .pem file permissions:
ssh -o "StrictHostKeyChecking=no" -i ${private_key} jumpuser@${JUMPBOXIP} "ls -la ~/.certs"
wait
Adjust the Permissions
On the machine you copied the certificates from, put the file permissions back to an acceptable range.
### 3. Certs permissions - VM from where you are copying the certificates and Azure Jump Box VM
#### 3.1 Change back file permissions, on your VM ~/.certs folder, after their copy on the jump box:
chmod 400 ~/.certs/gelazure_trustedcerts.pem
chmod 400 ~/.certs/gelazure_trustedcerts.bak
ls -la ~/.certs
#### if no longer needed on this machine, you can delete them from here. Do not delete them form the jump box.
#### 3.2 On the Jump Box VM:
##### Change the certificate permissions
ssh -o "StrictHostKeyChecking=no" -i ${private_key} jumpuser@${JUMPBOXIP} "sudo chmod 400 ~/.certs/gelazure_trustedcerts.pem"
wait
##### Set .certs folder permissions
ssh -o "StrictHostKeyChecking=no" -i ${private_key} jumpuser@${JUMPBOXIP} "sudo chmod 700 ~/.certs"
wait
echo List .pem file and .certs folder permissions
ssh -o "StrictHostKeyChecking=no" -i ${private_key} jumpuser@${JUMPBOXIP} "ls -la ~/.certs"
wait
ssh -o "StrictHostKeyChecking=no" -i ${private_key} jumpuser@${JUMPBOXIP} "ls -la ~"
wait
Conclusions
This post shown how to configure a Virtual Machine hosted on Azure to communicate with SAS Viya deployed on Azure. The first step is to establish Network Security Group rules that allow the communication. The second step is to obtain the SAS Viya .pem file from the SAS Viya Azure Kubernetes Cluster and copy it to the Virtual Machine.
After the configuration, you can use this virtual machine as an Azure Pipelines self-hosted agent. This will be the subject of the next post: Create an Azure DevOps self-hosted agent.
Thank you for your time reading this post. If you liked the post, give it a thumbs up! Please comment and tell us what you think about post content. If you wish to get more information, please write me an email.
