Deploying SAS Container Runtime models on Azure Container Instances
In this post we will look at using Azure Container Instances to run SAS Container Runtime (SCR) model images. The SAS Container Runtime is a lightweight Open Container Initiative (OCI) compliant container that provides a runtime environment for SAS models and decision flows.
Once the model or decision flow has been published to a container registry it is truly portable. It can run on a Docker or containerd runtime environment, Kubernetes cluster or one of the Cloud Providers serverless platforms. The Azure Container Instances (ACI) is Microsoft’s serverless platform.
Let’s look at deploying SAS Container Runtime models to ACI…
For simplicity when I refer to “SCR models” I mean analytical models and decision flows.
The illustration below provides a summary of the deployment options, depicting using SAS Model Manager to publish SAS models.
Select any image to see a larger version.
Mobile users: To view the images, select the "Full" version at the bottom of the page.
Before I get into the details of running SCR models on ACI, I want to take a moment to let you know about a key update that was provided in November (Stable 2022.11). As of Stable 2022.11 TLS support is now provided for the SCR model images. Prior to this only unencrypted access was supported. See the SAS Container Runtime documentation: Configuring TLS Security.
Azure Container Instances – what do you need to know?
The Microsoft documentation positions ACI as follows:
“Run Docker containers on-demand in a managed, serverless Azure environment. Azure Container Instances is a solution for any scenario that can operate in isolated containers, without orchestration. Run event-driven applications, quickly deploy from your container development pipelines, and run data processing and build jobs.”
See the Azure Container Instances documentation - serverless containers, on demand
So, is it a good fit for running the SCR models? I guess the short answer is “yes” or “it depends”!
I do think it is a good fit, the “it depends” comes down to the requirements and is a serverless platform (ACI) the best option.
Using Azure Container Instances
Deploying container images to ACI is a straightforward process, and like most things in the Azure cloud there are multiple options when it comes to deploying and configuring objects. You can use the Azure Portal GUI or one of the command line interfaces. In this article I will provide examples using the az command-line interface. The ‘az container create’ command is used to create an ACI container instance.
The ACI instance is associated with a resource group, and at a minimum the following key parameters must be specified:
- The name for the ACI instance.
- The name of the container image and any credentials to access (download) the image.
- The port to be used by the ACI container.
- Whether the ACI container will have a public IP address or whether it will run on a private network. If the container has a public IP address, you must also specify a DNS label. The DNS label is not used with private networking.
It is also possible to override the default for CPU and memory allocated to the ACI instance. But if increasing the CPU and memory allocations you must be mindful of the Azure Region you are using, as the limits do vary by region. At the time of writing this, the default in Azure EASTUS was 1 core with 1.5 GB memory. For more details see the following Microsoft documentation: Resource availability for Azure Container Instances in Azure regions.
As far as running SCR model images is concerned, the key consideration is that the Tomcat instance running in the SCR container is configured to use port 8080 by default, and port 8443 when TLS is configured. It is listening on port 8080 or 8443. However, it is not possible to remap port 8080 or 8443 as part of the ACI deployment. The ‘--ports’ parameter must be set to ‘8080’ or ‘8443’.
The following example is for running a model with a Public IP address, with public access. Therefore, a DNS label must be specified.
az container create
--resource-group my-resource_group-rg
--subscription ${SUBSCRIPTION}
--name scr-qstree1
--image myacr.azurecr.io/qs_tree1:latest
--registry-login-server ${ACR_SERVER}
--registry-username ${APP_CLIENT_ID}
--registry-password ${APP_CLIENT_SECRET}
--dns-name-label qstree1-xxxx
--ports 8080
In this example, the SCR model image was stored in the Azure Container Registry. An App Registration (Service Principle) is used to authenticate to the registry. The App ID and secret were store in variables.
Once the ACI instance has been created you can view it using the command-line or in the Azure portal. Using the command-line, the ‘az container show’ command gives the following output.
$ az container show --name scr-qstree1 --resource-group ${RG} -o table
Name ResourceGroup Status Image IP:ports Network CPU/Memory OsType Location
----------- -------------------- -------- -------------------------------- ------------------ --------- --------------- -------- ----------
scr-qstree1 my-resource_group-rg Running myacr.azurecr.io/qs_tree1:latest 52.152.247.22:8080 Public 1.0 core/1.5 gb Linux eastus
The following images are from my test environment. The first shows the resource group and the ACI instance, called ‘scr-qstree1’.
Looking at the ACI instance, you can see the DNS name that was created. This has a format of:
dns-label.region.azurecontainer.io
This probably isn’t the best deployment as the model is open to the world and the session traffic is unencrypted, it is not using TLS (HTTPS) encryption.
Therefore, a better approach is to deploy the SCR model image, the ACI instance, using secure private networking (and to configure TLS security). When using private networking the model would only be accessible from within the Azure Cloud, to surface the model a frontend load-balancer or an Azure Application Gateway could be used.
Using a frontend proxy (load-balancer or Azure Application Gateway) will allow the SCR port to be remapped, typically to port ‘80’ or ‘443’.
Using Private Networking
When using private networking you don’t define a DNS label, but you need to specify that the IP address is ‘private’ and the vnet and subnet to be used. For example, the following command would create an ACI instance using private networking:
az container create
--resource-group my-resource_group-rg
--subscription ${SUBSCRIPTION}
--name scr-qstree1
--image myacr.azurecr.io/qs_tree1:latest
--registry-login-server ${ACR_SERVER}
--registry-username ${APP_CLIENT_ID}
--registry-password ${APP_CLIENT_SECRET}
--ports 8080
--ip-address Private
--vnet my-vnet
--subnet my-aci-subnet
The VNET must be created before you can deploy the ACI instance. For example, this can be done using the ‘az network’ command:
az network vnet subnet create --name my-aci-subnet
--address-prefixes 192.168.3.0/24
--resource-group my-resource_group-rg
--vnet-name my-vnet
--network-security-group my-nsg The --address-prefixes specifies the CIDR range, in my environment I used 192.168.3.0/24. You can confirm the subnet creation using the following command:
az network vnet subnet list -g ${RG} --vnet-name ${vnet} -o table
In my environment this gave the following result. Note, my resource group had a number of subnets defined, the last one in the list was created for the ACI instance.
Conclusion
I hope you can see that it is very easy to deploy a SCR model image to ACI, but you do need to think about how the model(s) will be secured.
Finally, a limitation of using ACI is that there isn’t the concept of a replica sets. If a model needs to be highly available (maybe distributed across Availability Zones - not all regions support Availability Zones) or scaled to multiple instances for performance reasons, you must manage that yourself. This could be done by deploying multiple ACI instances and then configurating the frontend load-balancer to distribute the workload across the available ACI containers.
If the model(s) being published are mission critical, with service-level requirements mandating high availability and/or workload scalability, then it could be better to use a Kubernetes deployment.
Useful resources
- GitHub project: SAS Container Runtime (SCR) - the Low Footprint, High Performance Container for SAS Models
Thanks for reading. Michael Goddard
