How to Automate Model Deployment with SAS Viya and Azure DevOps

Learn how to import, register an analytical model in SAS Viya, publish and deploy automatically to Azure, where it will be ready for scoring requests, in less than eight minutes. Read on for the details.

Why SAS Viya and Azure Pipelines?

Stack Overflow wrote:

Data scientists excel at creating models that represent and predict real-world data, but effectively deploying machine learning models is more of an art than science. Deployment requires skills more commonly found in software engineering and DevOps. Venturebeat reports that 87% of data science projects never make it to production.

With SAS Viya and the Azure cloud platform you can tip the balance in your favor. Models that do not get deployed can become the exception, not the rule.

Watch the Pipelines in Action

In just under eight minutes, you can convert a dormant ZIP file into a registered model in SAS Model Manager, a container image, and a functional REST API that is prepared for scoring on Azure.

(view in My Videos)

What is Required

SAS Viya

SAS Viya Long Term Support version 2022.09, 2021.2 or 2021.1. For convenience, the SAS Viya in our examples is deployed in Azure.
SAS Model Manager licensed. You must create a project in SAS Model Manager prior to importing models.
An Azure publishing destination in SAS Viya. SAS Container Runtime (SCR) is a key component for converting a model into a container image.

Azure Cloud

Access to an Azure subscription and sufficient permissions.
An Azure Container Registry is required for the Azure publishing destination. See How to Publish a SAS Model to Azure with SCR: A Start-to-Finish Guide or Creating model publishing destinations using the SAS Viya CLI.
An Azure Web App. The web app serves as a target for the model deployment.

Azure DevOps

Access to an Azure DevOps Organization.
Create a release pipeline that deploys the model to an Azure Web App. For more information, refer to SAS Viya Model Release Pipeline with Azure DevOps.
A self-hosted agent pool with at least one agent that can “communicate” with your SAS Viya deployment. The self-hosted agent is a virtual machine and must have sas-viya CLI installed, Python, certificates copied, etc. Refer to the following resources for instructions:
An Azure pipeline that uses the self-hosted agent.
A Git repository.

How It All Works

(view in My Videos)

The Detailed Steps

The Model

Create and test an analytical model.
Export your model to a ZIP file.
As an alternative, you can use a zipped model form the SAS Model Manager Quickstart Tutorial.

The Git Repository

Upload the model ZIP file to a Git repository, such as Azure Repos, GitHub, GitLab, Bitbucket.
Create a requirements.txt file for Python packages. Insert just one word inside: sasctl. The sasctl package enables easy communication between the SAS Viya platform and a Python runtime. Read Sophia Rowland’s post or the python-sasctl SAS Software GitHub project for more information.

Register Model Program

Create a Python program called SCRModelRegister.py, that uses sasctl to register a model in SAS Model Manager from the ZIP file in your Git repository.

# Import necessary packages
import sys
import os
import requests
import shutil
from sasctl import Session
from sasctl.services import model_repository as mr, model_management as mm
import urllib3
import argparse

# Create an ArgumentParser object to handle command-line arguments
parser = argparse.ArgumentParser()
parser.add_argument('--host', type=str, required=True)
parser.add_argument('--user', type=str, required=True)
parser.add_argument('--password', type=str, required=True)
parser.add_argument('--protocol', type=str, required=True)
parser.add_argument('--repository_name', type=str, required=True)
parser.add_argument('--project_name', type=str, required=True)
parser.add_argument('--model_name', type=str, required=True)
parser.add_argument('--model_folder', type=str, required=True)
args = parser.parse_args()

# Print the command-line arguments
print('SAS Viya URL: ', args.host)
print('SAS User Name: ', args.user)
#print('SAS User Password: ', args.password)
print('Protocol: ', args.protocol)
print('SAS Model Repository Name: ', args.repository_name)
print('SAS Model Project Name: ', args.project_name)
print('SAS Model Name: ', args.model_name)
print('SAS Model Folder: ', args.model_folder)

# Set the path to the SSL certificates
pem_path='/home/jumpuser/.certs/my_trustedcerts.pem'

# Create a session using the specified arguments
s = Session(args.host, args.user, args.password, args.protocol, verify_ssl=pem_path)

# List object properties
print("Object doc: s")
print(dir(s))
print("Object doc: s._request_token_with_oauth")
print(dir(s._request_token_with_oauth))

# Get an access token using OAuth
at=s._request_token_with_oauth(args.user, args.password).access_token

# Retrieve a repository and project from SAS Model Manager
repository = mr.get_repository(args.repository_name)
print(f"Repository retrieved: {repository.name} with {repository.id}")

project = mr.get_project(args.project_name)
print(f"Project retrieved: {project.name} with {project.id}")

# Prepare to post the model to the specified URL
url = f"{args.host}/modelRepository/models"
print("Will POST the model", args.model_name, " to this url: ", url)
headers = {'Authorization': f'Bearer {s._request_token_with_oauth(args.user, args.password).access_token}'}

payload={
  'name': args.model_name,
  'type': 'GENERIC',
  'projectId': project.id
}
print("This model will be posted: ")
print(args.model_folder + "/" + args.model_name + ".zip")
files={'files' : open(args.model_folder + "/" + args.model_name + ".zip",'rb')}

# Post the model to the specified URL
register_model = requests.post(
    url,
    data = payload,
    headers = headers,
    files = files,
    verify=pem_path
)

# Print the content of the response
print(register_model.content)

Publish Model Program

Create a Python program called SCRModelPublish.py, that publishes the registered model from SAS Model Manager, to Azure, through the Azure publishing destination.

# Import necessary packages
import sys
import os
import requests
import shutil
from sasctl import Session
from sasctl.services import model_repository as mr, model_management as mm
import urllib3
import argparse

# Create an ArgumentParser object to handle command-line arguments
parser = argparse.ArgumentParser()
parser.add_argument('--host', type=str, required=True)
parser.add_argument('--user', type=str, required=True)
parser.add_argument('--password', type=str, required=True)
parser.add_argument('--protocol', type=str, required=True)
parser.add_argument('--repository_name', type=str, required=False)
parser.add_argument('--project_name', type=str, required=False)
parser.add_argument('--model_name', type=str, required=True)
parser.add_argument('--model_folder', type=str, required=False)
parser.add_argument('--dest_name', type=str, required=True)
parser.add_argument('--model_publish_name', type=str, required=True)
parser.add_argument('--model_publish_notes', type=str, required=True)
args = parser.parse_args()

# Print the command-line arguments
print('SAS Viya URL: ', args.host)
print('SAS User Name: ', args.user)
#print('SAS User Password: ', args.password)
print('Protocol: ', args.protocol)
print('SAS Model Repository Name: ', args.repository_name)
print('SAS Model Project Name: ', args.project_name)
print('SAS Model Name: ', args.model_name)
print('SAS Model Folder: ', args.model_folder)
print('Azure Publishing Destination Name: ', args.dest_name)
print('SAS Model Publish Name: ', args.model_publish_name)
print('SAS Model Publish Notes: ', args.model_publish_notes)

# Set the path to the SSL certificates
pem_path='/home/jumpuser/.certs/my_trustedcerts.pem'

# Create a session using the specified arguments
s = Session(args.host, args.user, args.password, args.protocol, verify_ssl=pem_path)

# Get an access token using OAuth
at=s._request_token_with_oauth(args.user, args.password).access_token

# Get the model from SAS Model Manager
model = mr.get_model(args.model_name)
print(model)

# Prepare to publish the model to the specified URL
headers = {
    "Content-Type": "application/vnd.sas.models.publishing.request.asynchronous+json",
    "Authorization": f"Bearer {s._request_token_with_oauth(args.user, args.password).access_token}"
}

# to publish model under a different name can use "modelName": args.model_publish_name
# to publish model under the same name can use "modelName": args.model_name

payload = {
    "destinationName": args.dest_name,
    "modelContents": [
        {
        "modelName": args.model_publish_name,
        "publishLevel": "model",
        "sourceUri": f"/modelRepository/models/{model.id}"
    }
    ],
    "name": args.model_publish_name,
    "notes": args.model_publish_notes
}

# Post/publish the model to the specified URL
model_publish = requests.post(
    url=f'{args.host}/modelManagement/publish?force=true',
    headers=headers,
    data= json.dumps(payload),
    verify=pem_path
)
# Print the content of the response
print(model_publish.content)

The Azure Pipeline

Create a pipeline in Azure DevOps that uses the Git repository.
Create a YAML script for the Azure pipeline that includes instructions to:
- Install pip on the agent machine.
- Install the packages defined in txt.
- Run the SCRModelRegister.py program with several arguments.
- Run the SCRModelPublish.py program with several arguments.
- Use sas-viya models CLI to list the SAS Model Manager models, published models, projects, and repositories.
Run or trigger the pipeline.

300_How_To_bt_1_Automate_Model_Deployment_with_SAS_Viya_and_Azure_DevOps_Import_Job-1024x463.png

Select any image to see a larger version.
Mobile users: To view the images, select the "Full" version at the bottom of the page.

Pipeline YML

# Python script importing a model from a .zip in SAS Viya Model Manager
# A second Python script publishes the model from SAS Viya Model Manager to Azure
# the SAS Model Manager repository and project must be created before
# the Azure publishing destination must be created before
# the pipeline passes parameters to the python scripts.

name: 06_051_Model_Automation
parameters:
- name: MODEL_NAME
  displayName: Model file name without the .zip at the end
  type: string
  default: QS_Reg1
trigger:
- main
pool:
  name: $(pool)
stages:
- stage: models
  displayName: Manage Models in SAS Model Manager
  jobs:
  - job: importModel
    displayName: Import Model in SAS Model Manager
    steps:
    - checkout: self
      persistCredentials: true

    - task: AzureKeyVault@2
      inputs:
        azureSubscription: '$(azureSubscriptionSC)'
        KeyVaultName: '$(keyvault)'
        SecretsFilter: '*'
        RunAsPreJob: true

    - script: |
        echo Check python
        python3 --version
        echo Python is installed
        echo Install pip3
        sudo apt install python3-pip -y
      displayName: 'Install Pip 3.x'

    - script: |
        echo 'List directory where Git Repo was cloned: ' && pwd
        ls -la
        echo Get the folder where the Git repo was cloned
        export GITFOLDER=$(pwd)
        echo Git Pull
        git pull origin main
        ls -la
        echo Install Requirements
        pip3 install -r $GITFOLDER/requirements.txt
      displayName: 'Install Python Requirements'

    - task: Bash@3
      inputs:
        targetType: 'inline'
        script: |
          echo 'List the folder where Git Repo was cloned: ' && pwd
          ls -la
          echo Get the folder where the Git repo was cloned
          export GITFOLDER=$(pwd)
          echo List the Python programs folder
          ls -la $GITFOLDER/programs
          echo List the models folder
          ls -la $GITFOLDER/programs
          echo Arguments to pass to the python script
          export INGRESS_URL=https://my_viya_url
          echo Viya URL: ${INGRESS_URL}
          SAS_USER=sas_user_here
          #SASPASS Will be retrieved from the Azure Key Vault $(keyvault) as a variable $(SASPASS)
          PROTOCOL=https
          REPO_NAME=Public
          PROJECT_NAME=QS_HMEQ
          echo Path to certificates - ADAPT PATH if NEEDED
          REQUESTS_CA_BUNDLE=~/.certs/my_trustedcerts.pem
          echo Execute Python Program with Parameters
          python3 $GITFOLDER/programs/SCRModelRegister.py --host $INGRESS_URL --user $SAS_USER --password $(SASPASS) --protocol $PROTOCOL --repository_name $REPO_NAME --project_name $PROJECT_NAME --model_name ${{ parameters.MODEL_NAME }} --model_folder $GITFOLDER/content > ~/SCRModelRegister.log
          echo List Execution Log
          cat ~/SCRModelRegister.log
      displayName: Import Model in SAS Model Manager

  - job: publishModel
    displayName: Publish Model to Azure
    dependsOn: importModel
    steps:
    - checkout: self
      persistCredentials: true
    - task: AzureKeyVault@2
      inputs:
        azureSubscription: '$(azureSubscriptionSC)'
        KeyVaultName: '$(keyvault)'
        SecretsFilter: '*'
        RunAsPreJob: true

    - task: Bash@3
      inputs:
        targetType: 'inline'
        script: |
          echo 'List the folder where Git Repo was cloned: ' && pwd
          ls -la
          echo Get the folder where the Git repo was cloned
          export GITFOLDER=$(pwd)
          echo List the Python programs folder
          ls -la $GITFOLDER/programs
          echo List the models folder
          ls -la $GITFOLDER/programs
          echo Arguments to pass to the python script
          export INGRESS_URL=https://my_viya_url
          echo Viya URL: ${INGRESS_URL}
          SAS_USER=sas_user_here
          #SASPASS Will be retrieved from the Azure Key Vault $(keyvault) as a variable $(SASPASS)
          PROTOCOL=https
          REPO_NAME=Public
          PROJECT_NAME=QS_HMEQ
          DEST_NAME=AzureCLI
          echo Path to certificates - ADAPT PATH if NEEDED
          REQUESTS_CA_BUNDLE=~/.certs/my_trustedcerts.pem
          echo Execute Python Program with Parameters
          python3 $GITFOLDER/programs/SCRModelPublish.py --host $INGRESS_URL --user $SAS_USER --password $(SASPASS) --protocol $PROTOCOL --model_name ${{ parameters.MODEL_NAME }} --dest_name $DEST_NAME --model_publish_name "sasmodel" --model_publish_notes "Published using REST API" > ~/SCRModelPublish.log
          echo List Execution Log
          cat ~/SCRModelPublish.log
      displayName: Publish Model from SAS Model Manager to Azure

  - job: listModels
    displayName: List Models in SAS Model Manager
    dependsOn: publishModel
    steps:
    - checkout: self
      persistCredentials: true
    - task: AzureKeyVault@2
      inputs:
        azureSubscription: '$(azureSubscriptionSC)'
        KeyVaultName: '$(keyvault)'
        SecretsFilter: '*'
        RunAsPreJob: true

    - script: |
        echo 'List the folder where the Git Repo was cloned: ' && pwd
        ls -la
        echo Get the folder where the Git repo was cloned in a variable
        export GITFOLDER=$(pwd)
        echo List the cloned Git Repo programs folder
        ls -la $GITFOLDER/programs
        echo Create SAS-Viya CLI Profile
        echo Initialize needed variables
        pwd && ls -la
        export SAS_CLI_PROFILE=gelenv
        echo $SAS_CLI_PROFILE
        export current_namespace=my
        export CURL_CA_BUNDLE=~/.certs/${current_namespace}_trustedcerts.pem
        echo $CURL_CA_BUNDLE
        export SSL_CERT_FILE=~/.certs/${current_namespace}_trustedcerts.pem
        echo ${SSL_CERT_FILE}
        export REQUESTS_CA_BUNDLE=${SSL_CERT_FILE}
        echo ${REQUESTS_CA_BUNDLE}
        export INGRESS_URL=https://my_viya_url
        echo Viya URL: ${INGRESS_URL}
        clidir=/opt/sas/viya/home/bin
        echo SAS-Viya CLI directory is: $clidir
        echo Create SAS-Viya CLI Profile
        $clidir/sas-viya --profile ${SAS_CLI_PROFILE} profile set-endpoint ${INGRESS_URL}
        $clidir/sas-viya --profile ${SAS_CLI_PROFILE} profile toggle-color on
        $clidir/sas-viya --profile ${SAS_CLI_PROFILE} profile set-output fulljson
        echo Show profile
        $clidir/sas-viya --profile ${SAS_CLI_PROFILE} profile show
        echo Login in
        SAS_USER=sas_user_here
        $clidir/sas-viya --profile ${SAS_CLI_PROFILE} auth login -user $SAS_USER -password $(SASPASS)
        echo *************** END COMMON TRUNK *******************
        echo Model Management Module
        echo Switching SAS Output to text
        export SAS_OUTPUT=text
        echo SAS Model Manager
        echo Get a List of Repositories
        $clidir/sas-viya --profile ${SAS_CLI_PROFILE} --output text models repository list
        echo Get a List of Projects
        $clidir/sas-viya  --profile ${SAS_CLI_PROFILE} --output text models project list
        echo List models
        $clidir/sas-viya --profile ${SAS_CLI_PROFILE} --output text models list
        echo Get a List of Publishing Destinations
        $clidir/sas-viya --profile ${SAS_CLI_PROFILE} --output text models destination list
        echo Get a List of Published Objects
        $clidir/sas-viya --profile ${SAS_CLI_PROFILE}  --output text models published list
      displayName: SAS Model Manager Lists

The pipeline requires several variables to be adapted:

MODEL_NAME: name of the model without extension.
pool: name of the self-hosted pool, configured to work with SAS Viya.
azureSubscriptionSC: the service connection authorizing the Azure DevOps project to access resources in an Azure subscription.
keyvault: the keyvault where secrets are stored.
SAS_USER:user.
INGRESS_URL: SAS Viya URL without / at the end.
current_namespace: Kubernetes namespace where SAS Viya is deployed.
SASPASS: sas user password stored in the keyvault above; retrieved at runtime.
REQUESTS_CA_BUNDLE: path to the SSL certificates. Should be the same as the pem_path in the Python programs.

The Release Pipeline

The pipeline publishes the model to Azure and triggers a release pipeline that deploys the model in a Web App. The release pipeline includes a trigger, artifact, and stage with a task to deploy the container image to an Azure web app when a new Docker container image sasmodel:latest arrives in the Azure Container Registry. For more information, refer to SAS Viya Model Release Pipeline with Azure DevOps.

Traceability

The model registration and publishing is captured in SAS Model Manager.

bt_2_310_How_To_Automate_Model_Deployment_with_SAS_Viya_Model_Manager_History-1024x349.png

The publishing captures the published container image name and the tag.

bt_3_320_How_To_Automate_Model_Deployment_with_SAS_Viya_Model_Manager_Deployment-1024x572.png

From this point forward, you have to look at the release pipeline logs in Azure DevOps or at the Azure Web App logs.

Conclusions

Before you know it, you can successfully import and register your analytical model in SAS Viya, and then have it published to Azure. It is amazing how quick and easy the entireprocess is. And you can start scoring your newly-deployed model straight away.

There are some serious benefits for data scientists everywhere using this method. It can help them deploy their models faster to production.

Acknowledgements

Thanks to my colleague @YiJianChing for helping out with sasctl.

Thank you for your time reading this post. If you liked the post, give it a thumbs up! Please comment and tell us what you think about Python and SAS Viya REST APIs. If you wish to get more information, please write me an email.

Find more articles from SAS Global Enablement and Learning here.