Azure spot instances are a cost-effective option provided by Microsoft Azure for running virtual machines (VMs) and workloads. Under traditional Azure virtual machine pricing, you pay a fixed rate. Spot instances allow you to take advantage of spare capacity in Azure’s data centers at significantly reduced prices. However, there’s a trade-off: Azure can reclaim the spot instance if the capacity is needed by paying customers. This therefore gives lower-cost compute resources but with the possibility of interruption. Wouldn’t it be great if we could use spot instances to run low-priority SAS Viya workloads? The key is to find workloads that can be restarted after an interruption without causing a problem.   The idea I wanted to test the idea of creating a “spot host type” within SAS Workload Management deployed on an Azure AKS service. This new host type would be required by a “spot” queue. If workload is submitted to the spot queue, Azure spot instances should be launched via Azure autoscaling. On Kubernetes, these Azure spot instances would have to be Kubernetes nodes running within an Azure spot node pool. We therefore start by creating an Azure spot node pool on the Azure Kubernetes service where we deployed SAS Viya.   Azure Spot Instances offer substantial cost savings, but they are not suitable for all workloads. Those that require uninterrupted, constant resources should be run on regular on-demand instances to avoid potential interruptions. When using Azure Spot Instances with SAS Viya, careful workload planning, job monitoring, and resource management are essential to maximize the benefits of cost savings and scalability, but manage the risk of interruption   Azure Spot node pool How to add an Azure Spot node pool is explained here. I used the command:   az aks nodepool add --resource-group hackinsas-viya202307-rg --cluster-name hackinsas-viya202307-aks -s Standard_L8s_v2 --name computespot8 --priority Spot --eviction-policy Delete --spot-max-price -1 --enable-cluster-autoscaler --min-count 0 --max-count 1 -c 1 --labels wlm=spot k8s\.azure\.com\/aks-local-ssd=true workload\.sas\.com\/class=compute launcher\.sas\.com\/prepullImage=sas-programming-environment --node-taints=workload\.sas\.com\/class=compute:NoSchedule --no-wait   If you set the max price to -1, the instance won't be evicted based on price. As long as there's capacity and quota available, the price for the instance will be the lower price of either the current price for a Spot instance or for a standard instance.The minimal node count is set to 0. That means that initially, you see one node created. However, shortly after you launch this command, the nodes will probably scale to 0. That might sound surprising, especially if you were expecting SAS compute pods to use this node to schedule workloads. The node is correctly labeled with “workload.sas.com/class=compute”, so in theory SAS compute pods could land here. However, spot instances have an additional label that stops scheduling: “kubernetes.azure.com/scalesetpriority=spot”. What do you need to do to be able to submit a SAS batch program to run on these instances?   Installing Kyverno We can use a tool called Kyverno to add the required toleration (“kubernetes.azure.com/scalesetpriority=spot”) to the SAS pods. Kyverno enhances Kubernetes by providing a policy-as-code framework that helps you enforce and manage various aspects of your workloads and resources in a Kubernetes cluster. Kyverno can be installed with these instructions. You can choose to install Kyverno with Helm or just download and apply an install.yaml:   kubectl create -f https://github.com/kyverno/kyverno/releases/download/v1.10.0/install.yaml   Adapting the SAS Workload Orchestrator and the SAS Image Staging Configuration DaemonSets We can use Kyverno to apply an extra toleration to the SAS Workload Orchestrator (SWO) Daemonset and to the SAS “prepull-ds” of the Staging Configuration process. An exampe yaml file below is available in the manifests folder. When pre-conditions are met, this code will mutate the selected resources. The mutation adds the missing toleration to two DaemonSets: the sas-workload-orchestrator DaemonSet and any DaemonSet starting with prepull-ds in its name.     kubectl apply -f ds-add-toleration-scalesetpriority.yaml -n <viya-namespace> Once this toleration has been added, sas-workload-orchestrator pods will be able to run on the spot nodes that were labeled with “workload.sas.com/class=compute”.  If the Kubernetes cluster starts any spot nodes in future, they will also automatically get sas-workload-orchestrator Daemonset pods. We have therefore used Kyverno to mutate the SWO daemonset to give it the necessary “kubernetes.azure.com/scalesetpriority=spot” toleration to get past the taint. It can now chase the “workload.sas.com/class=compute” label that was applied when provisioning the spot nodes. The node is now electable as candidate to host SAS compute workloads, and will be displayed as such in the workload orchestrator in SAS Environment Manager.   Looking at the details of the “computespot” host, all expected labels are set:     We added a toleration to any DaemonSet starting with “prepull-ds-*” because we want to let the SAS Image Staging process start required pods on spot nodes. That SAS Image Staging process will use a daemonset at a given time interval to ensure that relevant images have been pulled to hosts. Relevant images mainly means the sas-programming container. This will only be useful if at least one spot instance will run before you schedule workload on it. In practice, if the Azure autoscaler balances from 0 to n nodes, you will probably start with 0 nodes. After you submit workloads intended to run on spot instances, you will probably have to wait before the sas-programming container is pulled on the node. This reminds us again that spot instances should only be used for low priority jobs, when waiting five minutes doesn’t matter.   Adapting the SAS Workload Manager scaling pod. SAS Workload Orchestrator can integrate with the Kubernetes Cluster Autoscaler using this documentation. This means that the SAS Workload Orchestrator can use the Kubernetes Cluster Autoscaler to launch new compute nodes. To do this, a sas-workload-orchestrator-scaling-pod will be scheduled on a compute node. That scaling pod should run on a compute node with the correct resource properties specified via the SAS Workload management queue. We therefore need to know how to create a “spot queue”. Jobs submitted via a “spot queue” will only run on Azure spot instances. If no spot nodes are available, they can be autoscaled via a sas-workload-orchestrator-scaling-pod. However, as before, the scaling pods requesting a specific node can’t run on our spot nodes. We know why by now: they do not tolerate the spot taint. We need to create an extra Kyverno cluster policy via pod-add-toleration.yaml that will add that toleration to any scaling-pod in our Kubernetes Viya namespace.   kubectl apply -f pod-add-toleration-scalesetpriority.yaml -n <viya-namespace>     Creating new sas-batch-pod-template-spot podtemplate To make sure sas-batch-server pods hosting workloads meant for spot instances can run, we need to add the same extra scalesetpriority toleration needs to the podTemplate linked to the SAS Batch service launcher context. We need to create an extra sas-batch-pod-template for that. We can copy the original sas-batch-pod-template and modify the name and tolerations in the definition. An example yaml is available in the manifests folder.   kubectl apply -f sas-batch-pod-template-spot.yaml -n <viya-namespace> Adding spot host type to SAS WLM Creating a new SAS WLM host type is described in this documentation link. The host type created in our example here is named “spot”. In each host type, definition tags used to identify the hosts are specified. These tags can identify host characteristics that are relevant to SAS Workload Orchestrator. In the screenshot, the distinct tag used to identify Azure spot nodes is “wlm=spot”.   Adding spot queue to SAS WLM Rather like we created host types, we can create a queue “spot”. You can assign different hosts or host types to each queue. Each queue definition includes a list of the host types and tags that identify the hosts allowed to process jobs from the queue. If the queue definition includes tags, then when the SAS Workload Orchestrator manager evaluates which host to use to process a job from the queue, it checks the tag values on the queue and host, and sends the job to a host with a matching tag. By assigning the group of spot host type we created before, you can ensure that low-priority jobs will be processed on the Azure spot nodes.         Create new SAS Batch service launcher context spot The next step is creating a new SAS Batch service launcher context spot. From in SAS Environment Manager click on the new launcher context icon:     Fill in the details for the new launcher context as shown in the screenshot.     Creating new spot Batch context The final step is to create an extra Batch context. In the view where you defined the launcher context, switch to Batch contexts. Create a new one and name it ‘spot’. For the Launcher context, choose the one created in Step 7.     You can associate server contexts with SAS Workload Orchestrator queues. For information about associating a queue, see Contexts Page. Don’t forget to choose the dedicated “spot” queue as your SAS Workload Orchestrator queue.   Time for action As a test, you can launch workloads via a simple test script. Here is an example script that you can run from in a Linux session. You can submit any program, just make use of the parameter “c” that allows you to provide the context, which should be specified as “spot”. This will transfer the workload to a “spot” queue that is supposed to run only on Azure Spot Instances.   export SSL_CERT_FILE=/<ANY_DIR>/trustedcerts.pem INGRESS_URL=https://viya.sas.com /home/ubuntu/sas-viya --profile default profile set-endpoint "${INGRESS_URL}" /home/ubuntu/sas-viya --profile default profile toggle-color off /home/ubuntu/sas-viya --profile default profile set-output text /home/ubuntu/sas-viya --profile default auth login -u <user> -p <password> for i in {1..50} do echo "Welcome $i times" /home/ubuntu/sas-viya --profile default batch jobs submit-pgm -c spot --pgm prog1_hmeq.sas --restart-job /home/ubuntu/sas-viya --profile default batch jobs submit-pgm -c spot --pgm prog2_baseball.sas --restart-job /home/ubuntu/sas-viya --profile default batch jobs submit-pgm -c spot --pgm prog2_baseball.sas --restart-job done Before we kick off this shell script, we see four compute hosts. Looking closer, one is inactive. This is a spot compute host that is no longer part of the Kubernetes service. This may have been scaled down, possibly because no workload was submitted.   As soon the sas-workload orchestrator re-scans the Kubernetes services, inactive nodes are no longer displayed: Lets launch the shell script and see what happens. As soon the script is started, some simple SAS programs are submitted in batch: If we correctly set up the “spot” context, you will see the number of pending jobs rapidly growing in the “spot” queue: When there is no node available with the right criteria (meaning a Kubernetes node with a label wlm=spot), SAS Workload Management will launch a sas-workload-orchestrator-scaling pod and a Kubernetes node with the right criteria to schedule that pod.     The new node has the right criteria (it is tainted and labeled with “workload.sas.com/class=compute”) and SAS Workload Management will therefore recognize it as compute host. The sas-workload-orchestrator DaemonSet will launch a sas-workload-orchestrator pod on the Azure spot node and then the process will be confirmed, with the host displayed in the orchestration dashboard in SAS Environment Manager. Note status is OPEN-FULL, which is correct: immediately after starting the node, it will accept the maximum number of jobs permitted to start running.   However before the first programs can run, the sas-programming container image needs to be pulled onto the new compute node. This was explained and configured in Step 2, and in theory is done by the SAS Image staging process. However, there has not yet been time for that process.   We can confirm shortly after the first sas-batch-servers are scheduled that a prepull-ds pod is running on the aks-computespot node. The time cycle when these prepull pods are running can be configured.     Five minutes after the first SAS batch jobs were scheduled, they were running on the new compute nodes. Within those five minutes, the nodes were launched and all required pods were able to run on it. For a job that can “wait” and doesn’t require the highest priority, I think that’s a good result. This is especially true once you realize that these compute nodes can have discounts of up to 90 percent compared to pay-as-you-go prices.   The WLM dashboard confirms that the system is configured to run at most five jobs. All other jobs will stay in a pending state until a slot becomes available.   You can follow up the status of the submitted workload. This is important because programs can be interrupted if the spot instance is claimed back for a paying customer. You can automate this checking and also automatically restart the job if required.     When all goes well all your submitted workloads will be executed and all counters will be reset to 0 on the spot queue:   The autoscaler will then scale back the spot node pool to the minimum. In our example, that’s 0. We then go back to our start position, where the single aks-computespot in the WLM dashboard has a status of OPEN-INACTIVE.     Finally, if you check Kubernetes Nodes via kubectl or Lens, you will see confirmation that you have no aks-computespot hosts:   It means as well your batch job is completed.    Spot instances are a great way to get more benefit from the flexibility offered by your cloud vendor to reduce your overall cloud spent. While the above example is completely built for Azure, similar capabilities exist in both AWS and GCP.  This post illustrates the concept, to make it operate in a more customized production environment additional testing and configuration might be needed.     ... View more

Introduction In part1 of this series we created a customer connector via the Microsoft Power Automate platform (part1). We continued this series with a part2 where we created Power Automate Flow consuming the customer connector from part1.  Now let's integrate this Power Automate Flow in a Power App and demonstrate how SAS Viya REST endpoints can be consumed in our customer business applications.   Create a Power App Go to https://powerapps.com/. Let's start by creating an empty app.   You can start with adding Text labels:     After you can add 2 Text input fields next to each label. Make sure you switch format to Number.     To make next step easier change the name of those 2 input fields to 'limit' and 'start':     As next step add a button and label as you want.   t's now time to add the Power Automate flow we created. Click on the Power Automate and from there you can add your flow:     Now change the Function of the 'Fetch Rows' button to:     Then drag and drop the 'Data table (preview)' from the Layout section to the canvas:   Make sure you select the table you referenced in the function of the button. If you followed the example it should be 'tbl'. After that you can click 'Edit fields'   There you can click 'Add field' and select any number of fields that are returned by the PowerAutomate flow.   You should be able to preview the app.   ... View more

Introduction   On developer.sas.com it says that the SAS Viya REST APIs are designed for enterprise application developers, who intend to build on the work of model builders and data scientists, to deliver apps based on SAS Viya technology. With the SAS Viya REST APIs, you can create and access SAS resources using any third party application that supports http(s) calls.   Microsoft Power Platform connectors are wrappers or proxies around an API that allow the underlying service to talk to Microsoft Power Automate, Microsoft Power Apps, and Azure Logic Apps. SAS offers a SAS Decisioning Connector to run SAS Intelligent decisions that are deployed on your SAS Viya platform.   However Microsoft also gives you the option to build your own custom connectors which allows you to communicate with services that aren't available as prebuilt connectors. That way you could for instance connect from within PowerApps to a SAS model published in a SAS Scoring Container.   In this blog series that is split up in to three different parts, we will explain how you can create a Power App application that displays data retrieved from your SAS Viya environment. In the first part of the blog we will start with creating custom connector that list data loaded into the SAS Cloud Analytic Services (CAS)   Prerequisites The SAS Viya environment use TLS to secure data in motion.  You need a certificate from a commercial vendor like Digicert or Verisign.  Self signed certificates are not be trusted by the Power Automate of Power App platforms of Microsoft. And you are not able to import a self signed certificate.   In case the SAS Viya environment is protected by a firewall, you will need to whitelist the IP from which Power Automate requests are sent. More info about these IP addresses can be found here.   Create a Power Platform custom connector     Go to make.powerautomate.com and sign in to your Microsoft account. Click on New custom connector and select Create from blank.     Give the connector a name.     Now you can use a wizard that will guide you through the steps necessary to create your customer connector in Microsoft Power Automate. We will guide you through each of these steps and call out what needs to be setup for your custom connector.     General   Fill in your Viya hostname in the Base URL field.     Security   Now click on Security. Choose API key as authentication and fill the fields like you see below.     Definition   Click Definition and in the next screen you can determine actions that users will be able to perform. In our example we will only add 1 action: the retrieval of a number of rows from a CAS table. We start by clicking on New action and then we will define some general properties of the actions.     After that it's time to specify the details of the Request, for that you can click Import from sample.     Enter the following URL. We are going to modify the values between the brackets.   https://{viya-fqdn}/casRowSets/servers/cas-shared-default/caslibs/{cas-lib}/tables/{cas-table}/rows?limit=a&start=b   Parameter Description viya-fqdn The fully qualified domain name of the SAS Viya server. cas-lib The CAS library where the table is located. cas-table The CAS table name.   *) Leave the limit and start parameter to the respective values as shown above   Click Import.     Test   Before you are able to test the newly created customer connector you need to first save it by clicking on Create connector.     Once the custom connector has been saved we can start our test. But first we have to create a new connection. If no connection is listed click on "New connection"     To establish the connection to the SAS Viya environment you need to provide an API key. This API key is something that needs to be generated within the SAS Viya environment. How to generate this API key can be found here.        The API key needs to be in this format: Bearer <access_token>.  An example of what the API key could look like is shown here.   Bearer eyJhbGciOiJSUzI1NiIsImprdSI6Imh0dHBzOi8vbG9jYWxob3N0L1NBU0xvZ29uL3Rva2VuX2tleXMiLCJraWQiOiJsZWdhY3k tdG9rZW4ta2V5IiwidHlwIjoiSldUIn0.***********************6InVpZD12aX***********l9zaWciOiI4YTdjYzE4Ni0xZjE1L T***************************31HN-Upuhs-PKk7SEn_FWRyL0tasi0nNb_H0f****************VARJo67cpzyBxyv1gQg   You can validate if the connection was created successful by selecting connections:   You can go back to customer Custom connectors to test the connection. Click Test operation.   If the Access Token was generated correctly you will be able to run a successful test.     That concludes the creation for the custom connector. Now you're all set to create a Power Automate Flow which can be used in a later step in a Power App. We will explain how to do that in the second part of our blog series.   Let us know if you have any questions in the comments! ... View more

Kubernetes inherently comes with workload management controls. These are great for an IT administrator to maintain stability and harmony in their environment. These controls can extend to SAS. You may ask yourself, "since we have native workload management in our infrastructure, do I need more?" The answer is, "Yes."  In this article I'd like to introduce a new solution introduced by SAS, for SAS cloud deployments: SAS Workload Management.   Like my colleague @EdoardoRiva explains in his community article, Kubernetes provides workload management. In addition, he points out the latest release of SAS Viya is deployed on Kubernetes. Edoardo provides a high level overview of the advantages on SAS Workload Management. And in a very recent article Edoardo explains how the architecture is impacted when SAS WLM is added.  Having a curious mind, I was interested in discovering what SAS Workload Management brings in addition to the existing Kubernetes workload capabilities.   I wanted to see it live and experience myself how it all works, so let me explain how I configured a demo covering the essentials of SAS Workload Management.   By reading the SAS Viya Workload Management documentation I started to learn about pre-emption, queues, compute contexts, host-type, etc. Now let's put it all together.   Spin up Kubernetes Cluster with multiple compute nodes   I started with creating a SAS Viya Kubernetes environment in Azure by using the instructions in the viya4-iac-azure GitHub repository. In contrast with what I normally do (single node), I created an AKS cluster with four compute nodes in the SAS Compute Node Pool. I wanted to play with spreading workload over the different compute hosts and reserving a host for specific tasks. Figure 1 displays the organization of the cluster members.   Figure 1 - Kubernetes cluster organization for SAS Viya deployment   Remember, if you’re using the SAS IaC scripts to create your AKS cluster it’s very easy to configure a AKS cluster with four compute nodes.  Specify the min_nodes and max_nodes parameters in the Terraform.tfvars file as seen in Figure 2 below.   Figure 2 - min_ and max_nodes parameter settings in the Terraform.tfvars file   SAS Environment Manager & Workload Orchestrator   After deploying SAS Viya I navigate to the SAS Environment Manager, where I open the Workload Orchestrator. Consider the Dashboard tab in Figure 3 below.   Figure 3 - Workload Orchestrator Dashboard in SAS Viya Environment Manager   Figure 3 shows me I have three Queues available. I also get confirmation I have four compute servers available, which I can use to spread the SAS workload.   Checking the current SAS workload details on those queues is done via the Queue tab (Figure 4 below). Similar there’s a Hosts tab to see the current workload on all the hosts (Figure 5 below).   Figure 4 - Workload Orchestrator Queues   Figure 5 - Workload Orchestrator Hosts   Configuring Queues   Before launching any workload, let’s check the Configuration tab (Figure 6 below). Here we have a lot of options to define where the jobs submitted to a specific queue will run.   Figure 6 - Workload Orchestrator Configuration tab     Consider the Queue Configuration in Figure 7. First of all, you can assign a priority to the queue. Jobs submitted to queues with a higher priority will run before jobs submitted to queues with a lower priority.  Look back to Figure 4 and it's clear I have configured three queues in my demo environment: default, priority and interactive, respectively having priority 10, 20 and 30.   Figure 7 - Workload Orchestrator Queue Configuration   There’s also possibility to specify a list of other queues the queue can preempt.      Note that I selected for the interactive queue as Host Type SAS Studio interactive. This means for the interactive queue I would like to run the jobs that are submitted to this queue on hosts reserved purely for this type of work. More about configuring Host Types in the next section.     As my goal is to do a basic SAS Workload Management demo, so for now, that’s enough configuration for me. However, it should be clear there are many more to configurable queue parameters. For example, how many jobs can run per user and per host, who can submit jobs to the queue, or who can administer this queue.  Also, worth mentioning is the possibility to specify a completely new group of settings based on the time of the day.    Please  the SAS Workload documentation for further details.   Configuring Host Types   As mentioned earlier, I configured the interactive queue to run jobs on hosts that are of type SAS Studio Interactive.  These Host Types are configured by an administrator when selecting in the same configuration tab the Host Types (see Figure 8).   Figure 8 - Workload Orchestrator Hosts Configuration   For the SAS Studio Interactive Host Type, I assigned one specific compute node.  Here, I selected it by name. But in the field Host Properties, you can also make use of Kubernetes Node Labels.  I did that as well.  It should be in sync with a label on my compute nodes.   By running the following kubectl command I found my compute nodes and validated a "wlm" label exists.    kubectl get nodes -o=custom-columns=NODENAME:.metadata.name,LABELS:.metadata.labels | grep compute | grep wlm   As you can see in the results in Figure 9, three out four nodes have "wlm=default" but one is labelled with "wlm=interactive".  That’s the node I’m targeting for Host Type SAS Studio Interactive and where I would like to run jobs submitted to the interactive queue. I will discuss shortly how to configure so only SAS Studio jobs land & run on this interactive queue and associated compute host labelled "wlm=interactive".   Figure 9 - compute nodes   Lastly, I want to focus on another configuration difference between my default host type and the SAS Studio interactive host type on my demo environment (see Figures 10 and 11).   Figure 10 - default Host Type       Figure 11 - SAS Studio Interactive Host Type   I specified a different number for maximum jobs allowed. For the default hosts it’s set to four, whereas for the SAS Studio interactive I allow five jobs. During the demo it will help me to quickly discover which node is recognized as compute node to run SAS Studio jobs.   Configuring contexts   Let's now explore how to make sure a SAS Studio job is ending on the interactive queue and the associated compute node reserved for that queue.  Please refer to another article from Edoardo to learn all the details about the SAS runtime environment with the latest SAS Viya release on Kubernetes.  For now it’s important to know when requesting a SAS programming environment a Kubernetes SAS programming pod launches.  Depending how or from where the request is coming, a different context will be used: Compute context (with SAS Studio, SAS Model Studio, ..) Connect context  (with SAS/Connect) Batch context (when submitting program via sas-viya-client) Simply said, the program launches with a specific context, as seen in Figure 12.   Figure 12 - List of contexts   And with each compute, batch, or connect context a launcher context is associated, as seen in Figure 13.   Figure 13 - Launcher contexts SAS Studio compute context   Let’s take the example of SAS Studio: in Figure 14, a SAS Studio compute context will go together with SAS Studio Launcher context.   Figure 14 - SAS Studio compute and launcher contexts   And as seen in Figure 15, in the SAS Studio Launcher context, one specifies the SAS Workload Orchestrator queue. I assigned the interactive queue.  Remember jobs submitted to this queue will launch on computes nodes of host type SAS Studio interactive.   Figure 15 - SAS Studio launcher context   SAS viya-cli batch-context   As I will also use sas-viya-cli to launch programs from batch in the demo let me share how to configure the possibility to run priority batch jobs.    First, I go to the batch contexts. Next, to the predefined default context. I added an extra one here, a new, customized context named priority (see Figure 16). Figure 16 - Batch context   Checking the details, notice I specify in this context the SAS Workload Orchestrator queue. I assigned the priority queue as it has a higher priority than the default queue. I could have specified the Workload Orchestrator queue on the associated SAS Launcher context (that would be the SAS Batch Service Launcher context), but I preferred to make use of this Batch Context Workload Orchestrator field.    The bottom line is, I now have the choice to launch batch jobs with a default or priority context. As the default context  runs on the default queue, jobs submitted on that queue have to wait for an available slot on the compute nodes or might be even preempted by jobs with a higher priority.   Demo script   Time to see this in action. The following script launches SAS batch jobs with the  SAS Viya CLI.  I’m using three different profiles, each authenticating with a different user.   #!/bin/bash export SSL_CERT_FILE=/data/config/vdm/security/cacerts/** INGRESS_URL=https://myhost.sas.com /opt/sas/viya/home/sas-viya --profile default profile set-endpoint "${INGRESS_URL}" /opt/sas/viya/home/sas-viya --profile default profile toggle-color off /opt/sas/viya/home/sas-viya --profile default profile set-output text /opt/sas/viya/home/sas-viya --profile default auth login -u frederik -p password /opt/sas/viya/home/sas-viya --profile prod profile set-endpoint "${INGRESS_URL}" /opt/sas/viya/home/sas-viya --profile prod profile toggle-color off /opt/sas/viya/home/sas-viya --profile prod profile set-output text /opt/sas/viya/home/sas-viya --profile prod auth login -u edoardo -p password /opt/sas/viya/home/sas-viya --profile test profile set-endpoint "${INGRESS_URL}" /opt/sas/viya/home/sas-viya --profile test profile toggle-color off /opt/sas/viya/home/sas-viya --profile test profile set-output text /opt/sas/viya/home/sas-viya --profile test auth login -u sundaresh -p password123   After the authentication script, I loop over a set of commands, each launching SAS Programs.  Each loop launches two jobs with the default profile, one job with the test profile, and a series of jobs launch SAS programs with the production profile.  If you look carefully, you will see when using the prod profile I add "-c priority".  This means the request launches with the priority context of the batch context and consequently submits it to the priority queue.   for i in {1..50} do echo "Welcome $i times" /opt/sas/viya/home/sas-viya --profile default batch jobs submit-pgm -c default --pgm prog1_hmeq.sas --restart /opt/sas/viya/home/sas-viya --profile test batch jobs submit-pgm -c default --pgm prog2_baseball.sas --restart /opt/sas/viya/home/sas-viya --profile default batch jobs submit-pgm -c default --pgm prog2_baseball.sas --restart /opt/sas/viya/home/sas-viya --profile prod batch jobs submit-pgm -c priority --pgm prog2_baseball.sas --restart /opt/sas/viya/home/sas-viya --profile prod batch jobs submit-pgm -c priority --pgm prog3_baseball.sas --restart /opt/sas/viya/home/sas-viya --profile prod batch jobs submit-pgm -c priority --pgm prog4_baseball.sas --restart /opt/sas/viya/home/sas-viya --profile prod batch jobs submit-pgm -c priority --pgm prog5_baseball.sas --restart done   Queue and Host statuses   Soon after the initial loop runs, the priority queue will run the maximum of 12 jobs. A the same time the default queue has all jobs in pending state (see Figure 17).   Figure 17 - Queue Status   This is exactly what we expected to be happen because we defined max number of four running jobs and we also requested three out of four compute nodes for normal workload use.  So, once you have 12 jobs you have reached the maximum and jobs submitted to the priority queue will take precedence over the default queue.   Also, note that one host has nothing to do because we reserved it for SAS Studio sessions.  That’s why the first three hosts have status OPEN-FULL and the last one OPEN-OK (see Figure 18).   Figure 18 - Host Status  If a SAS Studio session is launched, a running program will be show on the interactive queue (see Figures 19 and 20).   Figure 19 - SAS Studio session   Figure 20 - Updated interactive queue Recorded demo   If you want to see a demo of the SAS Viya Workload Management, refer to the video below by my colleague Mark Schneider (@Mark_sas)  SAS Advisory Product Manager.  Mark talks about SAS WLM and introduces the demo described in this blog.  Have a look and let us know your thoughts.     ... View more

I think a lot of us know the feeling: you have a great idea and want to try it immediately on a SAS Viya environment.  So you need a deployment up and running in the next hour.  But then there's no server or nobody has the knowledge to install the software.   So before you know the momentum to execute your idea is over.   And yes I'm thinking about running an Analytics Sprint, or discussing a potential proof of concept in the meeting and starting with it the day after, or you want to test a brand new feature that has just been released with the latest SAS Viya cadence, or simple testing the connection of your environment to an IoT device.    In that case it would be very good to have something ready that automatically installs and configures such a SAS Viya deployment. The environment needs to be up and running quickly and consistent with the previous time you used it. Without human errors or mistakes that typically happen during a manual deployment. That's exactly what we developed here, and since we created this it already saved us a lot of time for use cases as described here above. How did we do it: by making use of Azure DevOps pipeline and installing the latest release on Azure AKS clusters.  Have a look to the following video to see it in action.         ... View more

This is part2 of a video recording how I perform a Blue/Green deployment on an existing demo Viya environment. My goal was to update Viya from stable 2021.1.3 to stable 2021.1.4. and transfer my end-users smoothly from the old version to the new version.     In part1 I recorded a video where I demonstrated the backup/restore process.  At the start of the next video I assume that the update has been done successfully by following the documentation.  So what's left is making sure that my users are now switched to the new Viya 2021.1.4 cadence.  The user will continue to use the same url but will be redirected to the new environment as soon we request a DNS switch of our INGRESS URL Hostname.  At SAS we can use internal application for that.   Watch the following video for the details how I did that. ... View more

As my colleague @RobCollum wrote in his post on communities.sas.com the combination of CI/CD and DevOps has led to a whole new world how software today is built, packaged and delivered.  Since Viya 2020 SAS is releasing software that way.  More specifically SAS Viya has two release cadences: Stable or Long-Term Support cadence.   Especially with the Stable cadence the updates come quickly: at least once a month and probably more when patches are released. While it’s nice to have continuous updates to the SAS software, production environments where users are actively creating content are often protected from frequent, ad-hoc changes.   That’s why I like the idea of a Blue/Green Deployment. I can make use of the backup/restore functionalities in SAS Viya to first create an identical box as my source environment.  Working this way we can say it's a Viya 4 to Viya 4 Full System migration.  But make sure you use the same Ingress Hostname.   After restoring the backup content I can safely apply the update on the new target environment.  If tests can confirm that everything is still working and all my Data, SAS Models, SAS VA reports, .. are still available I can execute the Blue/Green switch by changing the DNS entry of my Ingress Hostname to a new LoadBalancer IP address.   In this blog I wanted to share a video how I run the first part.  So how I restore a backup on a fresh Viya environment.   For the the update process I refer to the Viya Operation guide.  How to do the Blue/Green switch I will post in a follow up of this article but don't hesitate to drop me an email if you want to learn more about it. Let me start with introducing the flow that you will see in this video.   I will start with checking if a scheduled or ad-hoc backup completed successfully.  With the following kubectl get jobs command I can retrieve all backup jobs. (please note I’m using the alias k for kubectl)   k get jobs.batch -n hackinsas-eugtp -l "sas.com/backup-job-type=scheduled-backup" -L "sas.com/sas-backup-id,sas.com/backup-job-type,sas.com/sas-backup-job-status,sas.com/sas-backup-persistence-status" We will need the SAS-BACKUP-ID later when we do the restore of the content.   If you want to restore the backup content to a new target environment you will need to access the physical location of your Kubernetes Persistent volumes of the current Viya environment.  In my case the persistent volumes are provisioned via a NFSServer that I can access via a jumpbox. On that jumpbox I create two zip files (BACKUP_ID_cas.zip and BACKUP_ID_common.zip ) containing all the files I need to restore a backup.     The BACKUP_ID_cas.zip and BACKUP_ID_common.zip needs to be migrated to the similar Persistent Volumes of the target environment.  There you can unzip the files so that you have the backup available on the target environment.  Don’t forget to allow sufficient permissions to the extracted folders. Once  your BACKUP is on the target Persistent Volumes you can start following the ‘Perform a Restore’ like described here in the SAS Viya Operations guide.  (Make sure your select the guide for your release and cadence).   Watch the following video demonstrating backup and restore of SAS Viya content.     Big thanks to @RobCollum @GerryNelson @RPoumarede  for their feedback and help in creating this article. Also have a look to this articles from my colleague @GerryNelson:  a generic method for copying backup packages for migration or disaster recovery (introduced with 2021.1.4) a first look to backup and restore in SAS Viya on Kubernetes   ... View more

I was recently involved in work to design and set up the architecture for the SAS Global Hackathon, and especially setting up the SAS Viya sandboxes for all the teams. My main area of work—and interest—is analytics deployment, so it was interesting to see a rapid deployment and operational use in action. There were several user experience design factors that were considered in the deployment.  You can find a detailed technical description of my work done for the Hackathon in this blog but here’s what I learned from the experience.   Open source integration was a key requirement for users Users had some very important requirements. First, they needed open source integration with the sandbox. We provided them with code that would enable them to access the sandbox from Python or R-Code. We know that easy and robust integration of different technologies is essential for hackathons. However, it can also be important in everyday analytics deployments too. Data scientists often have different preferences for programming languages, but still need to be able to work together, and you need an environment that facilitates this.   It is essential to create trust in the environment We wanted hackathon participants to be able to get their data into the sandbox easily. Each team therefore had access to a Microsoft Azure Storage Account. We also carefully considered the rules for sandbox access, including firewall rules, whitelisting IP addresses, and authentication. These options meant that participants were confident about using the sandbox. This created a level of trust that was essential for success in the hackathon. This is very definitely a lesson that can be applied more widely: users need to be able to trust their environment and know that it is safe to use.   Flexibility can be embedded in several ways We designed three types of architecture to provide the right environment for different types of use case. This allowed us to support multiple use cases without having to create bespoke environments each time. To control costs, we switched the environments on and off automatically each day. However, we added flexibility by encouraging teams to email if they needed access at different times. We could then extend the time, or change the opening hours. The use of automated switching made life easier for the support team, and meant that we had time to provide the additional flexibility needed.   Self-service analytics provides multiple ‘wins’ for both users and organisations For part of the teams we opted for a self-service environment for many reasons. The first was pragmatic: with just a small team providing support, it was the most viable option to allow teams to do as much as possible for themselves. Analysts also often expect self-service, so it makes sense. However, another nice quick win from the self-service approach is self-education. We effectively gave hackathon participants a hands-on lesson in how a SAS Viya installation on an Azure Kubernetes Service cluster works. Self-service therefore provides faster learning—and so faster deployment—than other analytics environments. It also has the benefit that user acceptance is higher.   Automation, like self-service, provides several benefits for both users and organisations I mentioned the use of automatic switching to turn the environment on and off. We chose to automate several aspects of the environment for the hackathon, for several reasons. First because self-service tools for provisioning architecture are impossible without automation. We needed to be able to provide the environment without placing a big burden on IT operations. We also needed it to be both agile and efficient. Automation, especially with the cloud environment, offered both those aspects.   Speed of deployment can also be a crucial aspect of analytics architecture One of the key lessons for us—and for any architectures of analytics environments—was about the ease of providing SAS Viya sandboxes in the cloud. This is essential for a hackathon, where users need to get up and running fast. However, it would also be helpful to organisations or teams setting up analytical ‘sprints’ or demonstrations. It could therefore be used for conferences and events, or for students—or, indeed, many other uses across a wide range of organisations and sectors. I think one of the most useful aspects of supporting a hackathon is that we, as architects, can try new things. As my colleague James Ochiai-Brown commented on Twitter, “Our experience shows it is possible to set up self-service provisioning for the latest version of SAS Viya on Kubernetes. It really works!” At the SAS Global Hackathon, it was certainly not just the competitors who were experimenting or learning. ... View more

SAS is sponsoring a global hackathon, #HackinSAS. The event is designed to explore how to use data for social good in new and creative ways. I won’t go into details on the hackathon in this article, but I highly encourage you to refer to the #HackinSAS page as well as the Hacker’s Hub on the SAS Communities for more information. What I do want to cover in this post is how we went about creating the environment we’ll provide for each team participating in the hackathon.   Article Mission Introduce the possibilities Azure DevOps Pipelines bring in the context of automating SAS Viya deployments.   Pre-requisites This article relies heavily on two separate sassoftware GitHub repositories: SAS Viya 4 IaC for Azure SAS Viya 4 Deployment   These repositories contain tools, scripts, and code for SAS Viya deployment in an Azure environment. I highly recommend becoming familiar with these concepts.   Foundations I have been using the powerful SAS Viya Infrastructure as Code (IaC) repository (referenced above) to create the necessary Azure compute resources for a SAS Viya deployment.  The project contains Terraform scripts to provision Microsoft Azure Cloud infrastructure resources required to deploy SAS Viya 4 products. After the Infrastructure is created, I deploy SAS Viya with the code in the other referenced repository.     Azure DevOps pipelines The Azure infrastructure for #HackinSAS requires environments for 100 teams. Each team needs access to his own SAS Viya environment.  We are currently researching how to provide all that infrastructure in the most agile and efficient way, but I decided to start looking at Azure DevOps pipelines to automate this process as much as possible.   Azure Pipelines combine continuous integration (CI) and continuous delivery (CD) to constantly and consistently test and build your code and ship it to any target.  Consider it an automation server, similar to Jenkins.  My goal is to share with you how you can integrate the Viya4 IaC Terraform & Viya 4 deployment scripts in these pipelines. This requires some familiarity with automation servers and the SAS Viya IaC project.   The graphic below represents the entire deployment process explained. I’ll refer to it in the Pipeline initialization section.     Pipeline initialization Consider the first DevOps pipeline the initialization process before we can kick off the main pipeline.  It first creates a container to store the terraform.tfstate file within an existing Azure Storage Account.  Each participating team will have its unique terraform.tfstate file because it contains all the information about the Azure Cloud Infrastructure created for that team. The creation of that storage container is the first task of the pipeline.    The second task of the starters pipeline is to build a Docker Container Image, tag it with the name of the team and push it to an Azure Container Registry (step 3). This Docker image runs a few times in the second pipeline when we plan and apply the Terraform plan. Here are the Docker commands executed in the first pipeline to prepare the Docker image:   docker build -t viya4-iac-azure-hack:$(team) . docker tag viya4-iac-azure-hack:$(team) $(location).azurecr.io/viya4-iac-azure-hack:$(team) docker push $(location).azurecr.io/viya4-iac-azure-hack:$(team)   As you can see, we are making use of variables to identify team and locations.   The pipeline can't start without these required parameters. The Dockerfile (represented bellow) in use for the container image build is the same as what is found in the SAS IaC Github project. Note that one additional parameter,  ARM_ACCESS_KEY is required. This value provides access to the Azure Storage Account.   I also modified the Terraform init command by adding some backend parameters of the location of the terraform.tfstate file.  During the pipeline execution, the team string is replaced with the name of the hackathon team.   ... ENV ARM_ACCESS_KEY=__storagekey__ RUN apk --update --no-cache add git openssh \ && terraform init -backend-config="key=__team__-terraform.tfstate" -backend-config="container_name=__team__" /viya4-iac-azure ...   To get this working you also need to add the rest of the backend details to the main.tf of the Terraform scripts.   terraform { backend "azurerm" { resource_group_name = "tstate" storage_account_name = "tstatehackathon" } }   The pipeline execution (step 4) normally finishes in around two minutes. Below is a representation of the results of the pipeline execution.     Main pipeline Now it's time to do the bulk of the work which is setting up an Azure AKS Kubernetes cluster (step 5) and deploy Viya.  We set up the AKS cluster with the Docker image we pushed before to the Azure Container Registry. To deploy SAS Viya we make use of another very handy project on GitHub, SAS Viya4 deployment.   Just as with the first pipeline the idea is to check-in all necessary & parameterized files in an Azure DevOps project with an associated Azure Repos Git. azure_docker_creds.env terraform.tfvars ansible-vars-iac-azure.yaml TLS certificates   The first file contains the authentication information we will source during a specific stage in the pipeline.  It allow the Azure DevOps pipeline to authenticate Terraform to access our Azure subscription as described in the instructions for the SAS Viya 4 IaC for Azure project. Again, we need to add an additional variable to the azure_docker_creds.env file.  The ARM_ACCESS_KEY provides access the backend Azure storage so the Terraform state file can be saved in there.  Below is a representation of the file contents.   TF_VAR_subscription_id="00000000-0000-0000-0000-000000000000" TF_VAR_tenant_id="00000000-0000-0000-0000-000000000000" TF_VAR_client_id="00000000-0000-0000-0000-000000000000" TF_VAR_client_secret="00000000-0000-0000-0000-000000000000" ARM_ACCESS_KEY=__storagekey__   The pipeline queries the value of the storagekey with the appropriate credentials during the run.  In the terraform.tfvars and the ansible-vars-iac-azure.yaml I've added a few parameters to replace values during the run.   Below is a representation of the files.   terraform.tfvars # **************** REQUIRED VARIABLES **************** # These required variables' values MUST be provided by the User prefix = "hackinsas-__team__" location = "__location__" # e.g., "eastus2" # **************** REQUIRED VARIABLES ****************   ansible-vars-iac-azure.yaml ## Cluster NAMESPACE: hackinsas-__team__ .... JUMP_SVR_HOST: __jmpip__ JUMP_SVR_USER: jumpuser JUMP_SVR_PRIVATE_KEY: /data/auto-private-key-__team__.pem ... ## Ingress V4_CFG_INGRESS_TYPE: ingress V4_CFG_INGRESS_FQDN: __team__-hackinsas.vectorlabs.sas.com V4_CFG_TLS_MODE: full-stack # [full-stack|front-door|disabled] V4_CFG_TLS_CERT: /data/vector.crt V4_CFG_TLS_KEY: /data/vector.key V4_CFG_TLS_TRUSTED_CA_CERTS: /data/vector-cacerts.crt ..   With all of this in place, I kick off the Azure DevOps pipeline. This process takes about 30 minutes to complete.  During this time the setup of the infrastructure and the SAS Viya deployment is done automatically for me. This allows me to quickly spin up the necessary environments as teams are enrolling for the hackathon.  Here (graphic below) I start the pipeline manually but it can also be kicked off by using specific triggers.       When the execution completes, the pipeline creates an AKS cluster with six nodepools and kicks off a full SAS Viya deployment. About an hour later you can start working on the environment.   Details of the AKS cluster System node Pool default_nodepool_vm_type = "Standard_D4_v2" Node Pool Vm_type Min nodes Max Nodes system D4_v2 2 2 CAS E16ds_v4 1 1 compute E16ds_v4 1 1 connect E8ds_v4 1 1 stateless E16ds_v4 1 2 stateful E16ds_v4 1 3   As you can see from the screenshot below, I split the work in the pipeline over different stages and jobs.  Accept the challenge to provide more structure and better naming conventions 😊.  I also added the option to destroy the IaC and clean all the Azure resources when the Hackathon is finished.     And if you checked the details of the image below, you might see I'm publishing two artifacts.  The build artifact is a build-team.zip file that contains all the manifests, kustomization.yaml and site-config folder of the deployment.  The drop folder has the necessary files to connect to the Kubernetes cluster and Jump server created by the IaC terraform scripts. Finally, with that I can access and customize the SAS Viya environment if required.       Finally If this all is a bit abstract for you, I recorded a video demonstrating the Azure DevOps pipeline.  So, yes, we are getting ready for the Hackathon! ... View more

Earlier we discussed two ways to connect from SAS9.4 to Azure Quickstart. We demonstrated how to connect from a SAS94 client to the CAS controller in a SAS Viya Azure Quickstart Architecture.  As the CAS controller was in a private subnet, in the first article, we assigned a public IP address to the CAS controller and modified some network security inbound rules, allowing traffic from a well-known SAS94 network to the Azure Virtual Network.  In the second article we decided to step away from assigning a public IP address to the CAS controller and we added an extra Azure Loadbalancer to the Azure Quickstart architecture.   But maybe we don't want to add or remove any Azure resource to the Azure Quickstart and use the out of the box architecture:        So, the question is, if we use the Azure Application Gateway to access the CAS controller and submit CAS actions from the outside world, which port do we use?  The CAS port 5570 goes over a binary protocol, so that's not an option. We take into account HTTP or HTTPS are the only alternatives. The reason is that an Azure Application Gateway operates at the application layer (layer 7 – HTTP(S)).      Luckily, we can connect to CAS via REST HTTP calls.  Documentation around CAS REST APIs is available here.  In the previous articles the CAS actions I submitted in the sample code were loading data into the CAS engine from a SAS94 client.  So, the challenge that we must solve now is how to load data into our Viya CAS engine from a SAS94 client by sending a REST API Call.   The port that SAS Cloud Analytic Services listens to for HTTP(S) communication is commonly configured as port 8777.   So, it will come down to adding an extra Listener and associated Backend pool for the Application Gateway.  In the diagram below, you can see what we will add:         The added configuration allows an extra HTTPS connection.  As we will use HTTPS, it is necessary to configure the Listener and the Backend pool with the correct certificate. The instructions below are detailed, so keep the big picture of what we're describing in mind: setting up a secure connection to allow a client (SAS94) to communicate with our SAS Viya CAS controller.   New security rule All of this will only work by first configuring the PrimaryViyaLoadbalancer_NetworkSecurityGroup.  Add an extra inbound security rule for the IP range or address of your choice, as seen below.     Below are the rest of the steps you need to execute to get everything working.   Import the Azure certificate associated with the Application Gateway You may have noticed the ““Website Not Secure” message in your browser when doing a default Viya Quickstart installation.  This is by default as the Quickstart deployment generates a highly unique DNS name for your deployment and a self-signed certificate for secure connections. For limited use cases or proofs of concept we can ignore and accept the warning in the browser.  However, at the end of this guide we request an access token in some sample code by using the same URL in our SAS code.  Since it's included in code, we can't click and accept the risk of going to an insecure site.  We can work-around the issue by adding the generated certificate to a Windows trust store.  If you followed the other articles in this series, you seen this when importing the two SAS CA Certificates into the Windows Trusted Root Certificate Authorities Store.  Here are the instructions from the  SAS documentation.  Use same instruction for this certificate, but how can you find the certificate? We answer this below.   Find and copy the certificate You can find the certificate name by checking the appGatewayHTTPListener of the PrimayViyaLoadbalancer.   You should only see one Listener at this stage. It's configured to listen on port 443.        Double click on that line and you will find more details.     Notice there's a certificate named appGateWayFrontedCertificate. You can find the value of it by running the following command with the Azure CLI: az network application-gateway ssl-cert show --gateway-name PrimaryViyaLoadbalancer --resource-group *** --subscription *** --name appGatewayFrontendCertificate   This produces a JSON response resembling: { "data": null, "etag": "W/\"****\"", "id": "/subscriptions/****/resourceGroups/**/providers/Microsoft.Network/applicationGateways/PrimaryViyaLoadbalancer/sslCertificates/appGatewayFrontendCertificate", "keyVaultSecretId": null, "name": "appGatewayFrontendCertificate", "password": null, "provisioningState": "Updating", "publicCertData": "MIIGZwYJKoZIhvcNAQcCoIIGWDCCBlQCAQExADALBgkqhkiG9w0BBwGgggY8MIIGODCCBCCgAwIBAgI…..Taxqc/re5QLRVNCzR1MuxYLJLbf4tTtUVOZhpUNeetExgqLEB7i9LplBJ/UXq/vQwJst4br6BH+ND3j7CDMQA=", "resourceGroup": "***", "type": "Microsoft.Network/applicationGateways/sslCertificates" }   You need to copy what's returned in publicCertData key and save that to a .crt file. Import the new file into the Windows Trusted Root Certificate Authorities Store.     To check the previous step worked, go back to your browser to access your SAS Viya environment. Validate if the connection is secured:     Save the url in a text file, we will need it later during the test.   Create certificates for the CAS Backend pool and Listener. In future steps, we create a new Backend pool and a new Listener for the Azure Application Gateway.  As we will connect to the CAS HTTP 8777 port over HTTPS, we need to provide certificates. For the Backend pool we create a CER file and for the Listener a PFX file.  We use the openssl command on the ansible controller, but we need to use the correct input certificates you can find on the CAS controller. By default, in a full deployment like the Quickstart, the Security Certificates and Keys provided for CAS are in the following locations:     Connect to the controller with sftp and transfer sas_encrypted.crt and sas_encrypted.key to a temp work directory on the Ansible controller.  The screenshot below depicts this sequence.     You also need the value of the encryption key, so SSH to the controller and as root have a look at the contents of that file: cat /opt/sas/viya/config/etc/SASSecurityCertificateFramework/private/cas/shared/default/encryption.key   Create the certificates Now you can submit two OPENSSL commands to generate the necessary files. First, generate a controller.pfx file using the following command:   openssl pkcs12 -export -out controller.pfx -inkey sas_encrypted.key -in sas_encrypted.crt   and provide a personal export password when prompted. See the text below for the command line experience.       With the resulting controller.pfx file, create a certificate.cer file. openssl pkcs12 -in controller.pfx -out certificate.cer -nodes   Provide the password that you gave during the creation of the PFX file. See below.     Download certificate files Finally, download both files to your PC.     Before continuing, take note : in the following steps we update the Backend pools, Listeners, HTTP settings and certificates.  After applying changes, it can sometimes take a while (several minutes) before new settings are in place.  During that time, you can see the status "Updating". If that happens be patient and wait for testing or validating anything.     Configure the Backend pools Lookup the IP address of the CAS controller.      Take note of the IP address as we will need in a moment when we create a new Backend pool in the HTTP Application Gateway. Open the Application Gateway in the list of your Azure  Resources. From the properties window select the Backends pools:     Click the add button to create a new Backend pool.   Provide the private IP address of the CAS controller we checked at the beginning of this step.       The result will be a Backend without any rule associated, which is normal at this stage.     Configure the Listener Select your Application Gateway once more but now go to the Listeners:     Add a new Listener: make sure you use port 8777 and select the PFX file created in the previous step.  Give the certificate a meaningful name.  Don't forget to provide the import password previously set.     Add HTTP settings Now select in the Application Gateway properties window the HTTP settings:     Add a new HTTP Setting with port 8777, select HTTPS.  This is where you have to add the CER file.     Select the CER file and give it an appropriate name.     As a final step here set the Host name.  Choose pick host name from backend target.  That way the HTTP host header will be controller.viya.sas:     Add routing rule Again select the Application Gateway and go now to Rules:     Add a new rule and link the created Listener with the Backend pool and the HTTPSettings we created before.     Backend Health Before submitting any code, it's best to check at this stage the Backend health. You should now see two configured Backend Pools, and both should give a healthy status.       Test our SAS Code and if we can load data with a REST API call It's time to test if we can submit REST API calls.  In the other two articles around this topic an entry was created in the C:\Windows\System32\drivers\etc\hosts file.  In part one, we added an entry to point controller.viya.sas to the public IP of the CAS controller. Part two covered connecting to an Azure Load Balancer. Now we want to redirect traffic to the public IP of the Application Gateway:     Request access Token You will need CLIENT_ID and CLIENT_SECRET.  Have a look here if you need to register a SAS Viya Client ID and secret. The BASE_URI we are using is the DNS name of the public IP address associated with the Azure Application Gateway.  The rest of the code is also something I re-used from this SAS Commnunities article. ***************************************** *Use the Client ID to Get an Access Token; ****************************************** *Submit this code once to get the access token or repeat if your access token has expired.; options ls=max nodate; ods _all_ close; * Set the base URI for service calls, ; %let BASE_URI=https://viya-81a149c4…….h5kwsg.westeurope.cloudapp.azure.com; *%let BASE_URI=https://controller.viya.sas; * Specify the username and password; %let USERNAME=sasadmin; *replace YYY with user; *%let PASSWORD=Orion123; %let PASSWORD=myPassword; *MUST replace with an ACTUAL PASSWORD!; * Specify the Client ID name and secret; * Specify the new Client ID name; *! Name must be registered above - no spaces; %let CLIENT_ID=myApp; * Specify the secret for the new Client ID; %let CLIENT_SECRET=mySecret; * FILEREFs for the response and the response headers; filename resp temp; filename resp_hdr temp; * Get access and refresh tokens in JSON format; proc http url="&BASE_URI/SASLogon/oauth/token" method='post' in="grant_type=password%nrstr(&username=)&USERNAME%nrstr(&password=)&PASSWORD" username="&CLIENT_ID" password="&CLIENT_SECRET" out=resp auth_basic verbose; debug level=3; run;quit; *; * Get the access token from the JSON data and * store it in the ACCESS_TOKEN macro variable. *; libname tokens json "%sysfunc(pathname(resp))"; proc sql noprint; select access_token into:ACCESS_TOKEN from tokens.root; quit; %put This is the current value of the ACCESS_TOKEN: &ACCESS_TOKEN;   Create CAS session Now that we have access Token you can start a CAS session with PROC HTTP to the controller.viya.sas listening on port 8777. Remember controller.viya.sas is the Azure Application Gateway that will forward the request over to a Llistener on port 8777 that has a rule that is forwarding to the Backend pool, which is in this case our CAS controller. %let location=C:\temp; filename respa "&location\get_ref_a.json"; filename resphdra "&location\get_ref_a.txt"; %let CAS_URI=https://controller.viya.sas:8777/cas/sessions; proc http url="&CAS_URI" method='post' out=respa headerout=resphdra headerout_overwrite verbose; debug level=3; headers 'Authorization'="Bearer &ACCESS_TOKEN"; run;quit; libname sessions json "%sysfunc(pathname(respa))"; proc sql noprint; select session into:SESSION_ID from sessions.root; quit; %put This is the current value of the CAS SESSION_ID: &SESSION_ID;   Submit REST calls to load data We have a Viya CAS session and can now submit several CAS actions, but as our initial goal was to load into CAS from a SAS94 client let's do this here once more. filename respc "&location\get_ref_c.json"; filename resphdrc "&location\get_ref_c.txt"; %let CAS_URI=https://controller.viya.sas:8777/cas/sessions/&SESSION_ID/actions/upload; %put &CAS_URI; filename in "C:\temp\class.sas7bdat"; proc http url="&CAS_URI" method='put' in=in ct='binary/octet-stream' out=respc headerout=resphdrc headerout_overwrite verbose; debug level=3; headers "Authorization"="Bearer &ACCESS_TOKEN" "JSON-Parameters"='{ "casout": { "caslib": "public", "name":"class","promote":"true" },"importOptions": {"fileType":"basesas"} }'; run;quit;   Verify the results Check the GUI to see if the data is available:     Conclusion Once more we showed how we can load data from a SAS94 client to the SAS Viya CAS engine. That last one created by an Azure Quickstart template.  Instead of modifying the initial Azure Quickstart architecture or adding an extra Azure Load Balancer to the architecture we now used the existing Azure HTTP Application Gateway and make use of the fact that CAS can be accessed through REST API calls.   It comes down in configuring the Application Gateway with the correct Backend pools and Listener.  ... View more

In a previous article Connecting from SAS9.4 to Azure Quickstart, I showed how to connect from a SAS94 client to the CAS controller in a SAS Viya Azure Quickstart Architecture.  As the CAS controller was in a private subnet, we assigned it a public IP address and modified some network security inbound rules, allowing traffic from a well-known SAS94 network to the Azure Virtual Network.   That worked well but IT-departments might not like the idea of assigning a public IP address to an Azure VM running in a private Virtual Network.  So in case you followed my steps outlined in the first  article, let’s roll back and disassociate the public IP from the CAS controller.   Disassociate public IP address In the Azure console, navigate to the networking Settings of the controller Virtual Machine and click on Network Interface: CASController NetworkInterface.  In Network interface screen, click on IP configurations. You should see ipconfig1 with a Private IP address & Public IP address. Double click on ipconfig1.   Next, click the Disassociate button.  It’s good practice to take note of the private Virtual Network and private IP address:   Disassociate Public IP address   So, what now? As we no longer wish to expose our CAS controller, we need to find alternative Azure solutions.  One option is to employ an Application Gateway. In fact, such an Azure Application Gateway is part of the Azure Quickstart and takes the role of a PrimaryViyaLoadbalancer as you might remember from the environment configuration image below.   SAS Viya on Microsoft Azure The Azure Application Gateway operates at the application layer (layer 7 – HTTP(S)).  Although we can connect to CAS via REST HTTP calls, here the goal is to make a binary connection to CAS from a SAS94 client. I will show in a future article how to configure the Application Gateway in a way where you connect to the CAS controller with a REST HTTP CAS call.   Azure load balancer So with that in mind I switch gears and start looking to Azure load balancers.  While creating the load balancer in the Azure portal I had two choices:  the Standard Load Balancer or the Basic Load Balancer.  Check this comparison table for differences. I successfully tested both options, but I prefer the Standard Load Balancer as it’s built on the zero-trust network security model at its core.  The Standard Load Balancer is secure by default and is part of your virtual network as described in the image below.   Configure Azure load balancer   You can create a new public IP address or use an existing one.  This is the address you will use later to connect to the load balancer.   Create load balancer with a new IP address   Add backend pool Once the load balancer is created, the next step is to add a backend pool.  For simplicity we test using one single Azure VM, the CAScontroller. Select Backend pools from the left-hand menu of the Load Balancer settings.   Backend pools from Load Balancer Settings   Make sure you choose the correct Virtual network when adding the backed pool.  Choose a name that makes it easy to identify the backend (cascontroller as shown below).   Add backend pool configuration   When selecting the Virtual network, you also select the cascontroller as the backend VM.   Add cascontroller VM to backend pool   When complete with the steps, you should see something like the following.   Final backend pool configuration   Add health probes Before creating a load balancing rule to forward an incoming request to the right backend you will need to add a Health probe.  Click on the Health probes from the left-hand menu of the Load Balancer settings.   Select the Health probes setting   Choose an appropriate name and fill in port 5570 as default CAS port.  For this test you can keep the other values default.   Health probe settings configuration   Create load balancing rules Next, you create a load balancing rule by clicking on Load balancing rules from the left-hand menu of the Load Balancer settings.   Load balancing rules from Load Balancer Settings   Configuration of the rule is straightforward as we would like to distribute incoming traffic sent to the load balancer on port 5570 to the cascontroller backend pool with the same backend port.  Note, the only backend instances the health probe considers healthy, receive any traffic.  The screenshot below provides a sample of the configuration.   Load balaning rules configuration   Client-side configuration Finally, you will need to grab the public IP of the load balancer from Settings-> Frontend IP configuration.   Frontend IP Address   Copy the IP address and, for a test, override the entry of the controller.viya.sas in your C:\Windows\System32\drivers\etc\hosts file (replacing the ip address for your environment). If you followed part1 that entry should already exist (an entry similar to the text below). 2.4.81.158 controller.viya.sas   Important considerations In case you didn’t follow part1 you still have to check the network security group for an inbound port rule on port 5570.  Make sure it’s open for your IP address or CIDR range.   Also, before you can start submitting CAS statements to the CAS server from SAS 9.4 security certificates must be in place and you must be authorized to connect to the CAS server. For more information, see Configure SAS 9.4 Clients to Work with SAS Viya and Client Authentication Using an Authinfo File.    Or just follow the instructions in part1 for that, search for the sections “setting up the client authentication” and  “setting security certificates in place”.  When you re-submit the same sample code as in part1 you will connect to the CAS controller via the Load balancer.   Conclusion Instead of modifying the initial Azure Quickstart architecture and transferring the CAS controller from a private Azure virtual network to a public one, we are now keeping the CAS controller VM in its private network and are adding an extra Azure Load Balancer to the architecture.  It then comes down to configure the Load balancer with the correct backend. ... View more

The SAS Viya Azure Quickstart is a reference architecture for users who want to deploy the SAS platform, using microservices and other cloud-friendly technologies. By deploying the SAS platform in Azure, you get SAS analytics, data visualization, and machine-learning capabilities in an Azure-validated environment.   The following SAS Viya environment is built in the Microsoft Azure cloud.   SAS Viya on Microsoft Azure   As we can see, the CAS controller is in a private subnet and by default not accessible from the public internet.   Beginning with SAS 9.4M5, you can submit code to SAS Viya 3.3 and later releases of Cloud Analytic Services (CAS). From a SAS session with the CAS server, you can load data to CAS server memory. Then, you can save tables in CAS server memory and submit DATA step code, SAS Viya Analytic procedures, CAS server utility procedures, and Base SAS procedures.   Recently, I was contacted by a Danish colleague Christian Christensen who asked me how he could connect from a Windows on-premise SAS94 DI Studio client, to the CAS Viya controller in an Azure Quickstart, to load data.   This is accomplished in one of two ways. First, by creating a CAS libname that connects a SAS session to a CAS session. Or, you could start a CAS session directly in the SAS94 client and make use of PROC CAS. The issue with the Azure Quickstart is natively the CAS controller doesn’t have a public IP address and as such we can’t connect to the 5570 CAS port.   All is not lost, however. The process outlined in the steps below provide instructions on how to assign a public IP to the CAS controller for access to the SAS environment.   Make the CASController accessible Go to the networking Settings of the controller Virtual Machine and click on Network Interface: CASController NetworkInterface.   CASController network interface     In Network interface screen, click on IP configurations. You should see ipconfig1 with a Private IP address but not Public IP address. Double click on ipconfig1.   CASController IP configuration   Link a Public IP to the configuration by clicking on the Associate button.   Associate CASController public IP address   As we don’t want to use an existing IP address, click Create new.   Create new public IP address   Provide a name to the IP address. It’s good practice to make it a static IP.   Create static IP address   Next, associate this IP to the interface by clicking Save.   New public IP address created   If the association is successful, you should see a Public IP address associated to the CASController_NetworkInterface.   CASController network interface configuration   Now let’s test the connection. When we try to curl port 5570 with the newly created IP address it doesn’t work.   CAS connection attempt   The reason for the failure is we first must open port 5570 for the allowable IP address or Range.   Add port and source IP range   Note: as the user, you must fill in the Port and Source fields appropriate for your environment.   After configuring the port for the IP address, the curl to port 5570 should successfully resolve.   Successful CAS connection   Whereas we have cleared access to SAS, we are not still not ready to load data from our SAS9 client to CAS.  Before submitting programs to the CAS server from SAS 9.4, security certificates must be in place and you must be authorized to connect to the CAS server. For more information, see Configure SAS 9.4 Clients to Work with SAS Viya and Client Authentication Using an Authinfo File.   Setting up the Client Authentication Let’s start with creating the _authinfo file to authorize the client to connect to the CAS server. I followed an internal blog created by my colleague Gilles Chrzaszcz.   Here are the key steps: cd c:\Users\myuserid\ copy /y nul _authinfo icacls _authinfo /inheritance:r icacls _authinfo /remove “NT AUTHORITY\system” icacls _authinfo /remove “BUILTIN\Administrators” icacls _authinfo   Next, edit the _authinfo and make sure there’s an entry like this: host controller.viya.sas port 5570 user sasadmin password ****   Be sure your hostname is named controller.viya.sas because the certificate is created on that hostname. You can also use PROC PWENCODE to encode the password.    Also, add an entry to your C:\Windows\System32\drivers\etc\hosts file (replacing the ip address for your environment). 13.94.135.211 controller.viya.sas   Continuing on, let’s see if we can start a CAS session from our SAS client. options cashost="controller.viya.sas" casport=5570; cas mySession user=sasadmin; ERROR: Windows SSL error -2146869244 (0x80096004) occurred at line 2696, the error message is "The signature of the certificate cannot be verified. " ERROR: Secure communications error status 807ff008 description "40.113.77.78: Windows SSL error-2146869244 (0x80096004) occurred at line 2696, the error message is "The signature of the certificate cannot be verified. "" ERROR: Windows SSL error -2146869244 (0x80096004) occurred at line 2696, the error message is "The signature of the certificate cannot be verified. " ERROR: The TCP/IP negClientSSL support routine failed with status 807ff008. ERROR: Encryption run-time execution error ERROR: Failed to connect to host 'controller.viya.sas', port 5570. ERROR: Unable to connect to Cloud Analytic Services controller.viya.sas on port 5570. Verify connection parameters and retry.    As you can see from the error message, we need to import some certificates from SAS Viya. I highlight the process in the next section. You can follow the full instructions in the documentation:  Configure SAS 9.4 Clients to Work with SAS Viya.   Setting security certificates in place On the CAS controller navigate to the /opt/sas/viya/config/etc/SASSecurityCertificateFramework/cacerts directory. By using the keytool command (as seen below) I knew to use controller.viya.sas as the CASController hostname. It’s for this reason I added it in the hosts and _authinfo file on the client side. keytool -printcert -v -file trustedcerts.pem | grep DNSName GeneralName: DNSName: .gr GeneralName: DNSName: .eu GeneralName: DNSName: .edu GeneralName: DNSName: .org DNSName: localhost DNSName: controller DNSName: controller.viya.sas DNSName: *.controller.viya.sas DNSName: *.controller DNSName: services DNSName: services.viya.sas DNSName: *.services.viya.sas DNSName: *.services   To continue, let’s open the vault-services-ca.crt (located in the cacerts directory). An example of the file follows (output shortened for brevity). [sasdemo@sasserver cacerts]$ cat vault- services-ca.crt -----BEGIN CERTIFICATE----- MIIFJzCCAw+gAwIBAgIUT56HJhX9lBChchj+lnjOz0QYclAwDQYJKoZIhvcNAQEL .................. CIfCBnAMPnysgf1DFAyj9bva75oqJIE8n7lDwCUkAt/IU fEI0QxdhGAZTf4gnkIW2ioHavoKN8LLWY8hvdb5ylg2YdnimMIvUy30rJdcUEzFa o2vLH+ors9vVT7z86ZS2fk0uqS9h8dF5TLOp -----END CERTIFICATE----- -----BEGIN CERTIFICATE----- MIIEeDCCAmCgAwIBAgIUc+A1qJ32ANwuCxOR9YUp9qn8ciQwDQYJKoZIhvcNAQEL ................. rPNJidenyCDBjG0V7QegoQiy0UVGKUti1g2Ydn9juss+ii MQ/mYIJktWsc6+wlZesvWeEoToe6+lm9/Mc4WojfzNd0BxMEnK/Ga2vNGPY= -----END CERTIFICATE-----   The file contains two certificates:  the SAS Viya root CA certificate and the SAS Viya intermediate CA certificate. These files must be imported into the Window’s certificate store one at a time. This requires the creation of two unique files, one containing the root CA and the other containing the intermediate CA certificates. Use a text editor and cut and paste as appropriate.    Use the instruction from SAS documentation to import the two CA Certificates into the Windows Trusted Root Certificate Authorities Store.   Now it’s time to check if we can start a CAS session. /* STARTING CAS SESSION */ cas mySession user=sasadmin; cas mySession sessopts=(caslib=public); /*SUBMIT CAS LIBNAME*/ LIBNAME CASDM CAS CASLIB=PUBLIC PORT=5570 SERVER="controller.viya.sas" ; 17 /* STARTING CAS SESSION */ 18 options cashost="controller.viya.sas" casport=5570; 19 cas mySession user=sasadmin; NOTE: The session MYSESSION connected successfully to Cloud Analytic Services controller.viya.sas using port 5570. The UUID is c9a2cf49-fa10-9644-8024-36de2ce7403e. The user is sasadmin and the active caslib is CASUSER(sasadmin). NOTE: The SAS option SESSREF was updated with the value MYSESSION. NOTE: The SAS macro _SESSREF_ was updated with the value MYSESSION. NOTE: The session is using 0 workers. 20 cas mySession sessopts=(caslib=public); NOTE: 'Public' is now the active caslib. NOTE: The CAS statement request to update one or more session options for session MYSESSION completed.   Success!   And with that you have everything in place to load data directly into CAS.  You can use PROC CASUTIL: proc casutil; load data=sashelp.cars casout="cars1" promote; run; quit;   or, a SAS DATA step: data CASDM.CARS2; set sashelp.cars; run; proc casutil; promote casdata="cars2"; run; quit; cas mySession terminate user=sasadmin; Here is a repeat of the above steps with log information included: 24 proc casutil; NOTE: Writing HTML Body file: sashtml.htm NOTE: The UUID 'c9a2cf49-fa10-9644-8024-36de2ce7403e' is connected using session MYSESSION. 25 load data=sashelp.cars casout="cars1" promote; NOTE: SASHELP.CARS was successfully added to the "Public" caslib as "CARS1". 26 run; 27 quit; NOTE: PROCEDURE CASUTIL used (Total process time): real time 2.16 seconds cpu time 0.32 seconds 28 data CASDM.CARS2; 29 set sashelp.cars; 30 run; NOTE: There were 428 observations read from the data set SASHELP.CARS. NOTE: The data set CASDM.CARS2 has 428 observations and 15 variables. NOTE: DATA statement used (Total process time): real time 2.29 seconds cpu time 0.04 seconds 32 proc casutil; NOTE: The UUID 'c9a2cf49-fa10-9644-8024-36de2ce7403e' is connected using session MYSESSION. 33 promote casdata="cars2"; NOTE: Cloud Analytic Services promoted table CARS2 in caslib Public to table cars2 in caslib Public. NOTE: The Cloud Analytic Services server processed the request in 0.00117 seconds. 34 run;   You should see two datasets in PUBLIC Caslib.   Tables available in the Public Caslib   I’m now set up exactly the same way a local SAS Studio session allows me to connect to the Azure CAS controller.   By using the caslib _all_ assign statement I can see and work with the CASLIBS of the Azure Quickstart environment.   Assign all Caslibs in SAS Studio   Finally So, with that we were able to connect to a CAS instance which is in a private Azure Virtual Network.  The architecture of the Azure Quickstart has been well thought out and set up to allow only HTTPS traffic over the load-balancer.  But it’s good to know you have a quick alternative to connect directly to the CAS port controller, allowing your own on-premise SAS94 server to connect as a client. ... View more