How to Deploy and Score a SAS Model or Decision in Azure Kubernetes Service – Part 3

With SAS Container Runtime (SCR) you can publish a SAS rule set, decision, or a model in a container image. You can deploy the SCR image to Azure Kubernetes Service (AKS). AKS simplifies the implementation of Kubernetes clusters. It also provides all the orchestration features you need to manage cloud-native applications.

The three-part series shows you how to create and set up a simple AKS cluster, deploy and expose your scoring container to your users, through a deployment, a service and an ingress.

In this post, you will learn how to create an ingress, to expose internet traffic to the SAS Container Runtime service. Finally, you will learn how to score the deployed SAS decision or the model.

Enable Network Access to an Application

Create an Ingress

(view in My Videos)

What is an ingress? Ingress exposes routes for HTTP and HTTPS traffic from outside a cluster to services inside the cluster. You define ingress routes by using ingress rules. A Kubernetes cluster rejects all incoming traffic without these routes defined. For more information, see the following resources:

Why Ingress?

According to this source, ingress is probably the most powerful way to expose your services, but can also be the most complicated.

NodePort, which is used for Publishing Validation in SAS Viya, wins on simplicity. You need to open firewall rules to allow access to ports 30,000 to 32,767. In addition, you must know the IPs of the individual worker nodes. These IPs might change, especially if you perform scaling operations in your cluster. The node port is debated in another article.

Create an Ingress Manifest

To expose your service to the world via Domain Name System (DNS), you must create an ingress controller. Set the Fully Qualified Domain Name (FQDN) of the host allowed access to the cluster. In Cloud Shell, run the az network dns zone list command to query the Azure DNS zones.

az aks show \
  -g $RESOURCE_GROUP \
  -n $CLUSTER_NAME \
  -o tsv \
  --query addonProfiles.httpApplicationRouting.config.HTTPApplicationRoutingZoneName

In Cloud Shell, create a manifest file for the Kubernetes ingress.

touch scr-ingress.yaml

Open the integrated editor in Cloud Shell by entering code . . Open the YAML file, and add the following code. Do not forget to set the correct host with the FQDN retrieved above.

#scr-ingress.yaml
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: scoring
  annotations:
    kubernetes.io/ingress.class: addon-http-application-routing
spec:
  rules:
    - host: scoring...aksapp.io # Which host is allowed to enter the cluster
      http:
        paths:
          - backend: # How the ingress will handle the requests
              service:
               name: scoring # Which service the request will be forwarded to
               port:
                 name: http # Which port in that service
            path: / # Which path is this rule referring to
            pathType: Prefix # See more at https://kubernetes.io/docs/concepts/services-networking/ingress/#path-types

Deploy the Ingress

In Azure Cloud Shell, run the kubectl apply command to submit the ingress manifest to your cluster.

kubectl apply -f ./scr-ingress.yaml

The command should output a result similar to the following example.

ingress.networking.k8s.io/scoring created

Run the kubectl get ingress command to check if the deployment was successful.

kubectl get ingress scoring

The command should output a result similar to the following example.

NAME      CLASS    HOSTS                                                       ADDRESS         PORTS   AGE
scoring      scoring.49c0d56e92d8420c8268.australiasoutheast.aksapp.io   52.243.76.189   80      42m

Before continuing, the ADDRESS column of the output must be filled with an IP address. If it is not filled, wait a minute or so. To score the SCR container running in the pod, you need the DNS record created and linked to the ingress.

Score the SCR Decision or Model Running in the Kubernetes Pod

(view in My Videos)

In a browser, test if the SCR container responds via the Ingress. Add /__env at the end of the HOSTS from the previous log.

http://scoring.49c0d56e92d8420c8268.australiasoutheast.aksapp.io/__env

bt_2_908-SCR-container-address-in-an-AKS-pod-1024x93.png

The HOST + module = the endpoint that has to be used for scoring the SAS decision, rule set or model deployed to Kubernetes.

# Score a rule set in an Azure Kubernetes Service
# Generates Bid recommendation 0 or 1
curl --location --request POST 'http://scoring.49c0d56e92d8420c8268.australiasoutheast.aksapp.io/autoauction3_0'  --header 'Content-Type: application/json'  --header 'Accept: application/json'  --data '{
	"version":1,
	"inputs":[
	{"name":"BlueBookPrice","value":80000},
	{"name":"CurrentBid","value":90000},
	{"name":"Make","value":"Tesla"},
	{"name":"Miles","value":5000},
	{"name":"Model","value":"X100D"},
	{"name":"OriginalInvoice","value":100000},
	{"name":"OriginalMSRP","value":100000},
	{"name":"state","value":"CA"},
	{"name":"VIN","value":"12345678901234562"},
	{"name":"Year","value":2017}
	]
	}' | jq

The result resembles with the following output.

bt_3_910-SCR-container-scoring-in-an-AKS-pod-1024x585.png

Note: In the URL you do not need to specify the port 8080, as it was specified during the deployment. The defined service will redirect (internal) traffic from port 80 (http) to the container port 8080.

bt_4_915-SCR-container-scoring-service-and-ingress-in-an-AKS-pod-1024x653.png

With the URL and a sample scoring command, your users can now score the decision deployed inside the Azure Kubernetes Service cluster.

Conclusion

The third step was to create an Azure Kubernetes Service ingress, using manifest files. The ingress routes external traffic to the service within the cluster.

Finally, you can score a SAS decision, or a model, from a SAS Container Runtime image deployed to an Azure Kubernetes Service pod.

As a reminder the post series treated these steps:

Create and set up an AKS cluster.
Deploy Kubernetes workloads with deployment YAML files.
Expose Kubernetes workloads with service and ingress YAML files.
Score your SAS decision or the model running in a pod.

Discussion

The method discussed is not the only method for deploying a SAS decision or model to an AKS pod. It has advantages and shortcomings. It might not be appropriated for production environments. The main advantage is the time to implement it.

The article was written not to set best practices, but to share an approach. Therefore please share your thoughts, if you see any shortcomings, or have seen a better way.

Acknowledgements

My colleague at SAS, David Estreich for encouraging me to explore this path.

SAS Documentation

SAS Communities and YouTube

Azure Resources

Thank you for your time reading this article. If you liked the article, give it a thumbs up! Please comment and tell us what you think about SAS Container Runtime and Azure deployments.