SAS Viya CLI container image in Azure DevOps (March 2024 Update)

1 Like

In the latest development in the realm of containerization, the SAS Viya command-line interface (CLI) has made its debut in a container image. This milestone brings with it a host of advantages, offering streamlined batch operations within SAS Viya Platform deployments or functioning autonomously as a standalone container. For a comprehensive overview, delve into the intricacies outlined in the SAS Viya Platform: Using the Command-Line Interface documentation available here: SAS® Help Center - What’s New in Administration - 2024.03 (March 2024) - Command-Line Interfaces

On the same subject, Gerry Nelson will be releasing soon an article describing an "Overview of installing and using the SAS Viya containerized CLI". This article might be a good introduction of this new component prior jumping into customizing it for DevOps purposes.

This post marks a significant update from my previous entries, "Unlocking Efficiency with Container Jobs in Azure DevOps" and "Streamlining SAS Viya Authentication in Azure DevOps". Noteworthy changes include a shift towards employing a Linux-based client, and, most notably, leveraging the newly available sas-viya-cli image included to any SAS Viya 2024.03 deployment.

01_DE_Streamlining-SAS-Viya-Authentication-in-Azure-DevOps-1024x366.png

Source: Streamlining SAS Viya Authentication in Azure DevOps

Select any image to see a larger version.
Mobile users: To view the images, select the "Full" version at the bottom of the page.

For those familiar with my prior writings, the procedural steps to integrate the SAS Viya CLI into an Azure Container Job may ring familiar:

Retrieve Your Credentials:

- Authenticate to cr.sas.com to gain access to and utilize the new sas-viya-cli container image.
- (Optional) Consider pushing the sas-viya-cli container image into your Azure Container Registry for added convenience.
- Authenticate to your SAS Viya platform and procure your access and refresh tokens.

Push Your Credentials:

- Upload your credentials.json file into your Microsoft Azure Key Vault.

Utilize Your Credentials:

- Build an updated container image tailored for deployment within the container job via the Azure DevOps pipeline.
- Push the resultant image to your designated Azure Container Registry.
- Reflect the necessary adjustments in your Azure DevOps pipeline.
- Execute your Azure DevOps pipeline.

In the interest of clarity and brevity, this post refrains from delving into additional configurations, reserving them for a forthcoming follow-up post. Topics slated for discussion include links to security enhancements, encompassing considerations such as:

Azure KeyVault IP restrictions
Azure Kubernetes Cluster API IP restrictions
Kubernetes Ingress IP restrictions
Use / Revocation of the Refresh-Token
Certificates not chained to a public Certificate Authority

Stay tuned for the subsequent post, where I will elaborate on these critical aspects. Now, let's delve into the intricacies of each step involved in integrating the SAS Viya CLI into Azure DevOps container jobs.

Authenticate to cr.sas.com be able to pull & use the new sas-viya-cli container image

In the pursuit of integrating the sas-viya-cli container image into your workflows, a crucial initial step entails authenticating to cr.sas.com to allow pulling the new image. Detailed instructions for these actions can be found in the SAS Help Center: Command-Line Interface SAS Help Center: Command-Line Interface documentation, specifically under the section titled "Instructions When Running the CLI from a Container".

Below, I outline the requisite commands, assuming you have already downloaded the SAS Mirror Manager and obtained your order certificates, ensuring a smooth progression through this authentication process:):

# Set some variables
MIRRORMGR_PATH=$HOME

# Run the SAS Mirror Manager command to get cr.sas.com credentials
CR_COMMAND=$($MIRRORMGR_PATH/mirrormgr list remote docker login --deployment-data $MIRRORMGR_PATH/SASViyaV4_XXXXXX_certs.zip)

# Extracting username using grep and awk
DOCKER_CR_USERNAME=$(echo $CR_COMMAND | grep -oP -- '-u \K\S+')

# Extracting password using grep and awk
DOCKER_CR_PASSWORD=$(echo $CR_COMMAND | grep -oP -- "-p '\K[^']+")

docker login -u "$DOCKER_CR_USERNAME" -p "$DOCKER_CR_PASSWORD" cr.sas.com

(Optional) Push the sas-viya-cli container image into your Azure Container Registry

In scenarios where authentication via the cr.sas.com container registry is not feasible or permitted due to network restrictions, an alternative option involves pushing the sas-viya-cli container image into your Azure Container Registry. This ensures accessibility and control over the image within your Azure environment.

Before proceeding, ensure you've logged in to Azure using the az login command and authenticated to cr.sas.com using the previous step commands.

Here are the step-by-step instructions, tailored for execution within a Linux Command prompt (bash):

# az login
SOURCE_DOCKER_REGISTRY=cr.sas.com
TARGET_DOCKER_REGISTRY=myownacr

TARGET_DOCKER_ACCESS_TOKEN=$(az acr login --name $TARGET_DOCKER_REGISTRY --expose-token | jq -r '.accessToken')
docker login $TARGET_DOCKER_REGISTRY.azurecr.io -u 00000000-0000-0000-0000-000000000000 -p $TARGET_DOCKER_ACCESS_TOKEN

az acr import --force -n $TARGET_DOCKER_REGISTRY \
  --username $DOCKER_CR_USERNAME --password $DOCKER_CR_PASSWORD \
  --source $SOURCE_DOCKER_REGISTRY/viya-4-x64_oci_linux_2-docker/sas-viya-cli:latest

Authenticating to Your SAS Viya Platform and Retrieving Access & Refresh Tokens

In preparation for running SAS Viya CLI from an Azure DevOps environment, the initial step involves retrieving your SAS Viya credentials locally before uploading them to an Azure Key Vault.

Below are the step-by-step instructions, tailored for execution within a Linux Command prompt (bash):

# Set some variables
DOCKER_REGISTRY=cr.sas.com

export SAS_SERVICES_ENDPOINT=https://azureuse011070.my-trials.sas.com

# docker run --rm -it -e JWT_TOKEN -e SAS_SERVICES_ENDPOINT $DOCKER_REGISTRY/viya-4-x64_oci_linux_2-docker/sas-viya-cli:latest auth loginCode
docker run --rm -it -v $PWD:/cli-home/.sas -e SAS_SERVICES_ENDPOINT $DOCKER_REGISTRY/viya-4-x64_oci_linux_2-docker/sas-viya-cli:latest auth loginCode

Upon executing these commands, you'll obtain your access and refresh tokens. Subsequently, you'll have two authentication options within your Azure DevOps pipeline:

Option 1: Pass the Access token as the JWT_TOKEN variable. This approach is valid until the access token expires, typically within a short timeframe defined by the sas.logon.jwt.policy.accessTokenValiditySeconds parameter (default: 1 hour). While efficient, this method may not be suitable for lengthy DevOps pipeline executions.

Option 2: Rebuild the credentials.json file using both the access and refresh tokens. This method remains valid until the refresh token expires, determined by the sas.logon.jwt.policy.refreshTokenValiditySeconds parameter (default: 14 days). This approach offers greater longevity, making it potentially more suitable for prolonged DevOps pipeline executions.

Option 1 will be detailed in this post, option 2 will be detailed the follow-up post.

Uploading Your Credentials to Azure Microsoft KeyVault

Having obtained a valid JSON Web Token (JWT) and a refresh token, the next step is to securely upload these credentials to your Azure Key Vault. This ensures centralized management and enhanced security of your sensitive authentication data.

Even if option 1 only require uploading the JWT_TOKEN, for flexibility and in anticipation to use option 2, I opted to upload the entire credentials.json file, incorporating both the access and refresh tokens along with their respective expiry dates, as a single object into your Azure Key Vault. This consolidated approach eliminates the need to manage these tokens separately, thereby streamlining the authentication process within your Azure environment while ensuring comprehensive security and compliance.

Below are the instructions for executing this task within a Linux Command prompt (bash):

# az login (prerequisite)
# Set some variables
AZURE_KEYVAULT_NAME=SAS-COMMUNITIES-KV
AZURE_SECRET_NAME=SAS-COMMUNITIES-CREDENTIALS

# Create a secret (if one doesn't exist) or update a secret in a KeyVault.
az keyvault secret set --vault-name $KV --name $CR --file $PWD/credentials.json --output none

Upon execution, your credentials will be securely stored within your Azure Key Vault, safeguarding them against unauthorized access and ensuring seamless integration with your Azure DevOps pipeline.

Enhancing Container Image Creation for Azure DevOps Pipeline

In my previous post, we manually installed the SAS Viya CLI during the Docker image build process. However, in this iteration, we will optimize our workflow by utilizing the new sas-viya-cli image as the foundation for our updated container job image. The only additional component we incorporate is nodejs, a prerequisite for images used in Azure Container Jobs.

Below is the Dockerfile showcasing the enhancements:

FROM cr.sas.com/viya-4-x64_oci_linux_2-docker/sas-viya-cli:latest

USER 0

RUN mv /etc/yum.repos.d/ubi.repo /etc/yum.repos.d/ubi.repo.sas \
  mv /etc/yum.repos.d/ubi.repo.orig /etc/yum.repos.d/ubi.repo \
  rm /etc/yum.repos.d/sas*

RUN dnf install -y --nodocs nodejs && dnf clean all

ENTRYPOINT [ "" ]

CMD [ "node" ]

Please note that this example employs cr.sas.com as the container registry. However, if you followed the optional step, you could utilize your own registry during the build process.

Now, let's proceed with the creation of the updated container image. Execute the following commands in your Linux Command prompt (bash):

# Set some variables
TARGET_DOCKER_REGISTRY=myownacr.azurecr.io
VERSION=1.0

# Build locally the new Container Job docker image and tag it to be pushed to your own Azure Container Registry
docker build . -t $TARGET_DOCKER_REGISTRY/viya-4-x64_oci_linux_custom/cj-sas-viya-cli:$VERSION-$(date +%Y%m%d.%s) -t $TARGET_DOCKER_REGISTRY/viya-4-x64_oci_linux_custom/cj-sas-viya-cli:latest

The resulting image will then be tagged appropriately for utilization within your Azure DevOps pipeline and subsequently pushed to your designated Azure Container Registry.

Publishing Your Local Image to Azure DevOps Pipeline via Azure Container Registry

Now that you've successfully built your Docker image locally, the next step is to make it accessible to your Azure DevOps pipeline by hosting it on your own Azure Container Registry (ACR). This ensures seamless integration and deployment within your pipeline environment.

Below are the instructions to execute within a Linux Command prompt (bash):

# Set some variables
TARGET_DOCKER_REGISTRY=myownacr
TARGET_DOCKER_ACCESS_TOKEN=$(az acr login --name $TARGET_DOCKER_REGISTRY --expose-token | jq -r '.accessToken')

# Log in to the Azure Container Registry to be able to push the image
docker login $TARGET_DOCKER_REGISTRY.azurecr.io -u 00000000-0000-0000-0000-000000000000 -p $TARGET_DOCKER_ACCESS_TOKEN

docker push -a $TARGET_DOCKER_REGISTRY.azurecr.io/viya-4-x64_oci_linux_custom/cj-sas-viya-cli

Updating Your Azure DevOps Pipeline for Enhanced Integration

Now, we'll walk through the necessary steps to reflect the changes introduced, ensuring smooth deployment of your SAS Viya CLI commands in your Azure DevOps pipeline.

Defining Parameters for Enhanced Flexibility

To start, let's define the parameters required for your pipeline. These parameters provide flexibility and customization options tailored to your specific environment:

parameters:
- name: SAS_VIYA_URL
  displayName: Specify the URL for the SAS Viya platform environment
  type: string
  default: https://sasviyarg.gelenable.sas.com
- name: AZURE_KEYVAULT
  displayName: Specify the Azure KeyVault you want to use
  type: string
  default: SAS-COMMUNITIES-KV
- name: AZURE_KEYVAULT_KEY
  displayName: Specify the Secret you want to use
  type: string
  default: SAS-COMMUNITIES-CREDENTIALS

These parameters allow you to customize the pipeline according to your SAS Viya platform URL and Azure KeyVault configuration, providing greater control and adaptability.

Updating the Pipeline Structure

With the parameters defined, let's update the pipeline structure to reflect the latest changes. We'll introduce two jobs: one to retrieve the credentials from KeyVault and another to execute your SAS Viya CLI commands.

Retrieving Your Credentials from Azure KeyVault

Now, we'll guide you through the process of retrieving your credentials.json file from Azure KeyVault.

Here is the yaml extract for the job to execute this step:

- job: getJWTfromAKV
  displayName: Get SAS Viya Access Token
  steps:
  - task: AzureCLI@2
    name: TgetJWTfromAKV
    displayName: Get JSON Web Token from Azure Key Vault
    inputs:
      azureSubscription: '$(KV_SERVICE_CONNECTION)'
      scriptType: 'bash'
      scriptLocation: 'inlineScript'
      inlineScript: |
        az keyvault secret download --name ${{ parameters.AZURE_KEYVAULT_KEY }} --vault-name ${{ parameters.AZURE_KEYVAULT }} --file $PWD/credentials.json
        export JWT_TOKEN=$(cat $PWD/credentials.json | jq -r '.Default."access-token"')
        echo "##vso[task.setvariable variable=JWT_TOKEN;isOutput=true]$JWT_TOKEN"

This job, titled "Get SAS Viya Access Token," employs the AzureCLI task to extract the JSON Web Token (JWT) from Azure KeyVault. Here's a breakdown of the steps:

Download Credentials: Utilizing the Azure CLI, the script downloads the credentials.json file from Azure KeyVault and saves it to the pipeline workspace.
Extract JWT Token: The script extracts the access token from the downloaded credentials.json file using jq.
Set Output Variable: Finally, the access token is stored as an output variable, enabling its use in subsequent pipeline steps.

Notice that this step is NOT using a container job but uses a “standard” job where the Azure CLI is available by default.

Executing Your Azure DevOps Pipeline

Now, we'll guide you through the process of running your Azure DevOps pipeline, leveraging the SAS Viya CLI commands for streamlined execution.

Here is the yaml extract for the job to execute this step:

- job: runViyaClilogin
  dependsOn: getJWTfromAKV
  displayName: Run the SAS Viya CLI commands
  variables:
  - name: JWT_TOKEN
    value: $[ dependencies.getJWTfromAKV.outputs['TgetJWTfromAKV.JWT_TOKEN'] ]
  container:
    image: $(AZURE_ACR).azurecr.io/viya-4-x64_oci_linux_custom/cj-sas-viya-cli:1.0-20240329.1711733600
    endpoint: 'ACR Connection'
  env:
    SAS_SERVICES_ENDPOINT: ${{ parameters.SAS_VIYA_URL }}
    JWT_TOKEN: $(JWT_TOKEN)
  steps:
  - task: Bash@3
    displayName: Retrieve all configuration definition list
    inputs:
      targetType: 'inline'
      script: |
        cd /opt/sas/viya/home/bin
        /opt/sas/viya/home/bin/sas-viya-wrapper --output text configuration configurations list

This job, titled "Run the SAS Viya CLI commands," orchestrates the execution of SAS Viya CLI commands within your Azure DevOps pipeline. Here's an overview of the steps involved:

Dependencies: This job depends on the successful execution of the previous job, ensuring that the SAS Viya access token is retrieved from Azure KeyVault beforehand.
Define Variables: The access token retrieved from KeyVault is assigned to the JWT_TOKEN variable, facilitating secure authentication for SAS Viya CLI commands.
Container Configuration: The job runs within a specified container image hosted in your Azure Container Registry (ACR), ensuring consistency and reproducibility of the execution environment.
Environment Configuration: Environment variables are configured to specify the SAS Viya platform URL and pass the JWT token for authentication.
Execution Steps: The SAS Viya CLI commands are executed within the pipeline, enabling tasks such as retrieving configuration definition lists for seamless management of your software solutions.

Conclusion

In this post, we've introduced the new SAS Viya Command-Line Interface (CLI) container image for Azure DevOps pipelines. This update streamlines SAS Viya Platform deployments, offering efficiency and flexibility. We began by outlining the transition to a Linux-based client and leveraging the sas-viya-cli image. Then, we integrated the CLI into Azure DevOps pipelines, covering credential retrieval from Azure KeyVault, Docker image optimization, and pipeline updates. Executing the pipeline, we seamlessly ran SAS Viya CLI commands within our containerized environment.

Stay tuned for deeper dives into configurations and security enhancements in future posts. Our containerization journey continues with more insights and innovations.

Additional Azure DevOps Resources with SAS Viya

David Estreich (myself) related posts with Azure DevOps:

Bogdan Teleuca's other posts about SAS Viya with Azure DevOps:

Find more articles from SAS Global Enablement and Learning here.