Connecting from SAS9.4 to Azure Quickstart - part 3 - connect to CAS via REST HTTP calls
Earlier we discussed two ways to connect from SAS9.4 to Azure Quickstart. We demonstrated how to connect from a SAS94 client to the CAS controller in a SAS Viya Azure Quickstart Architecture. As the CAS controller was in a private subnet, in the first article, we assigned a public IP address to the CAS controller and modified some network security inbound rules, allowing traffic from a well-known SAS94 network to the Azure Virtual Network. In the second article we decided to step away from assigning a public IP address to the CAS controller and we added an extra Azure Loadbalancer to the Azure Quickstart architecture.
But maybe we don't want to add or remove any Azure resource to the Azure Quickstart and use the out of the box architecture:
So, the question is, if we use the Azure Application Gateway to access the CAS controller and submit CAS actions from the outside world, which port do we use? The CAS port 5570 goes over a binary protocol, so that's not an option. We take into account HTTP or HTTPS are the only alternatives. The reason is that an Azure Application Gateway operates at the application layer (layer 7 – HTTP(S)).
Luckily, we can connect to CAS via REST HTTP calls. Documentation around CAS REST APIs is available here. In the previous articles the CAS actions I submitted in the sample code were loading data into the CAS engine from a SAS94 client. So, the challenge that we must solve now is how to load data into our Viya CAS engine from a SAS94 client by sending a REST API Call.
The port that SAS Cloud Analytic Services listens to for HTTP(S) communication is commonly configured as port 8777. So, it will come down to adding an extra Listener and associated Backend pool for the Application Gateway. In the diagram below, you can see what we will add:
The added configuration allows an extra HTTPS connection. As we will use HTTPS, it is necessary to configure the Listener and the Backend pool with the correct certificate. The instructions below are detailed, so keep the big picture of what we're describing in mind: setting up a secure connection to allow a client (SAS94) to communicate with our SAS Viya CAS controller.
New security rule
All of this will only work by first configuring the PrimaryViyaLoadbalancer_NetworkSecurityGroup. Add an extra inbound security rule for the IP range or address of your choice, as seen below.
Below are the rest of the steps you need to execute to get everything working.
Import the Azure certificate associated with the Application Gateway
You may have noticed the ““Website Not Secure” message in your browser when doing a default Viya Quickstart installation. This is by default as the Quickstart deployment generates a highly unique DNS name for your deployment and a self-signed certificate for secure connections. For limited use cases or proofs of concept we can ignore and accept the warning in the browser. However, at the end of this guide we request an access token in some sample code by using the same URL in our SAS code. Since it's included in code, we can't click and accept the risk of going to an insecure site. We can work-around the issue by adding the generated certificate to a Windows trust store. If you followed the other articles in this series, you seen this when importing the two SAS CA Certificates into the Windows Trusted Root Certificate Authorities Store. Here are the instructions from the SAS documentation. Use same instruction for this certificate, but how can you find the certificate? We answer this below.
Find and copy the certificate
You can find the certificate name by checking the appGatewayHTTPListener of the PrimayViyaLoadbalancer. You should only see one Listener at this stage. It's configured to listen on port 443.
Double click on that line and you will find more details.
Notice there's a certificate named appGateWayFrontedCertificate. You can find the value of it by running the following command with the Azure CLI:
az network application-gateway ssl-cert show --gateway-name PrimaryViyaLoadbalancer --resource-group *** --subscription *** --name appGatewayFrontendCertificate
This produces a JSON response resembling:
{
"data": null,
"etag": "W/\"****\"",
"id": "/subscriptions/****/resourceGroups/**/providers/Microsoft.Network/applicationGateways/PrimaryViyaLoadbalancer/sslCertificates/appGatewayFrontendCertificate",
"keyVaultSecretId": null,
"name": "appGatewayFrontendCertificate",
"password": null,
"provisioningState": "Updating",
"publicCertData": "MIIGZwYJKoZIhvcNAQcCoIIGWDCCBlQCAQExADALBgkqhkiG9w0BBwGgggY8MIIGODCCBCCgAwIBAgI…..Taxqc/re5QLRVNCzR1MuxYLJLbf4tTtUVOZhpUNeetExgqLEB7i9LplBJ/UXq/vQwJst4br6BH+ND3j7CDMQA=",
"resourceGroup": "***",
"type": "Microsoft.Network/applicationGateways/sslCertificates"
}
You need to copy what's returned in publicCertData key and save that to a .crt file. Import the new file into the Windows Trusted Root Certificate Authorities Store.
To check the previous step worked, go back to your browser to access your SAS Viya environment. Validate if the connection is secured:
Save the url in a text file, we will need it later during the test.
Create certificates for the CAS Backend pool and Listener.
In future steps, we create a new Backend pool and a new Listener for the Azure Application Gateway. As we will connect to the CAS HTTP 8777 port over HTTPS, we need to provide certificates.
For the Backend pool we create a CER file and for the Listener a PFX file. We use the openssl command on the ansible controller, but we need to use the correct input certificates you can find on the CAS controller.
By default, in a full deployment like the Quickstart, the Security Certificates and Keys provided for CAS are in the following locations:
Connect to the controller with sftp and transfer sas_encrypted.crt and sas_encrypted.key to a temp work directory on the Ansible controller. The screenshot below depicts this sequence.
You also need the value of the encryption key, so SSH to the controller and as root have a look at the contents of that file:
cat /opt/sas/viya/config/etc/SASSecurityCertificateFramework/private/cas/shared/default/encryption.key
Create the certificates
Now you can submit two OPENSSL commands to generate the necessary files. First, generate a controller.pfx file using the following command:
openssl pkcs12 -export -out controller.pfx -inkey sas_encrypted.key -in sas_encrypted.crt
and provide a personal export password when prompted. See the text below for the command line experience.
With the resulting controller.pfx file, create a certificate.cer file.
openssl pkcs12 -in controller.pfx -out certificate.cer -nodes
Provide the password that you gave during the creation of the PFX file. See below.
Download certificate files
Finally, download both files to your PC.
Before continuing, take note : in the following steps we update the Backend pools, Listeners, HTTP settings and certificates. After applying changes, it can sometimes take a while (several minutes) before new settings are in place. During that time, you can see the status "Updating". If that happens be patient and wait for testing or validating anything.
Configure the Backend pools
Lookup the IP address of the CAS controller.
Take note of the IP address as we will need in a moment when we create a new Backend pool in the HTTP Application Gateway. Open the Application Gateway in the list of your Azure Resources. From the properties window select the Backends pools:
Click the add button to create a new Backend pool. Provide the private IP address of the CAS controller we checked at the beginning of this step.
The result will be a Backend without any rule associated, which is normal at this stage.
Configure the Listener
Select your Application Gateway once more but now go to the Listeners:
Add a new Listener: make sure you use port 8777 and select the PFX file created in the previous step. Give the certificate a meaningful name. Don't forget to provide the import password previously set.
Add HTTP settings
Now select in the Application Gateway properties window the HTTP settings:
Add a new HTTP Setting with port 8777, select HTTPS. This is where you have to add the CER file.
Select the CER file and give it an appropriate name.
As a final step here set the Host name. Choose pick host name from backend target. That way the HTTP host header will be controller.viya.sas:
Add routing rule
Again select the Application Gateway and go now to Rules:
Add a new rule and link the created Listener with the Backend pool and the HTTPSettings we created before.
Backend Health
Before submitting any code, it's best to check at this stage the Backend health. You should now see two configured Backend Pools, and both should give a healthy status.
Test our SAS Code and if we can load data with a REST API call
It's time to test if we can submit REST API calls. In the other two articles around this topic an entry was created in the C:\Windows\System32\drivers\etc\hosts file. In part one, we added an entry to point controller.viya.sas to the public IP of the CAS controller. Part two covered connecting to an Azure Load Balancer. Now we want to redirect traffic to the public IP of the Application Gateway:
Request access Token
You will need CLIENT_ID and CLIENT_SECRET. Have a look here if you need to register a SAS Viya Client ID and secret. The BASE_URI we are using is the DNS name of the public IP address associated with the Azure Application Gateway. The rest of the code is also something I re-used from this SAS Commnunities article.
*****************************************
*Use the Client ID to Get an Access Token;
******************************************
*Submit this code once to get the access token or repeat if your access token has expired.;
options ls=max nodate;
ods _all_ close;
* Set the base URI for service calls, ;
%let BASE_URI=https://viya-81a149c4…….h5kwsg.westeurope.cloudapp.azure.com;
*%let BASE_URI=https://controller.viya.sas;
* Specify the username and password;
%let USERNAME=sasadmin; *replace YYY with user;
*%let PASSWORD=Orion123;
%let PASSWORD=myPassword;
*MUST replace with an ACTUAL PASSWORD!;
* Specify the Client ID name and secret;
* Specify the new Client ID name;
*! Name must be registered above - no spaces;
%let CLIENT_ID=myApp;
* Specify the secret for the new Client ID;
%let CLIENT_SECRET=mySecret;
* FILEREFs for the response and the response headers;
filename resp temp;
filename resp_hdr temp;
* Get access and refresh tokens in JSON format;
proc http url="&BASE_URI/SASLogon/oauth/token" method='post'
in="grant_type=password%nrstr(&username=)&USERNAME%nrstr(&password=)&PASSWORD"
username="&CLIENT_ID" password="&CLIENT_SECRET" out=resp auth_basic verbose;
debug level=3;
run;quit;
*;
* Get the access token from the JSON data and
* store it in the ACCESS_TOKEN macro variable.
*;
libname tokens json "%sysfunc(pathname(resp))";
proc sql noprint;
select access_token into:ACCESS_TOKEN from tokens.root; quit;
%put This is the current value of the ACCESS_TOKEN: &ACCESS_TOKEN;
Create CAS session
Now that we have access Token you can start a CAS session with PROC HTTP to the controller.viya.sas listening on port 8777. Remember controller.viya.sas is the Azure Application Gateway that will forward the request over to a Llistener on port 8777 that has a rule that is forwarding to the Backend pool, which is in this case our CAS controller.
%let location=C:\temp;
filename respa "&location\get_ref_a.json";
filename resphdra "&location\get_ref_a.txt";
%let CAS_URI=https://controller.viya.sas:8777/cas/sessions;
proc http url="&CAS_URI"
method='post'
out=respa headerout=resphdra headerout_overwrite verbose;
debug level=3;
headers 'Authorization'="Bearer &ACCESS_TOKEN";
run;quit;
libname sessions json "%sysfunc(pathname(respa))";
proc sql noprint;
select session into:SESSION_ID from sessions.root; quit;
%put This is the current value of the CAS SESSION_ID: &SESSION_ID;
Submit REST calls to load data
We have a Viya CAS session and can now submit several CAS actions, but as our initial goal was to load into CAS from a SAS94 client let's do this here once more.
filename respc "&location\get_ref_c.json";
filename resphdrc "&location\get_ref_c.txt";
%let CAS_URI=https://controller.viya.sas:8777/cas/sessions/&SESSION_ID/actions/upload;
%put &CAS_URI;
filename in "C:\temp\class.sas7bdat";
proc http url="&CAS_URI"
method='put'
in=in
ct='binary/octet-stream' out=respc headerout=resphdrc headerout_overwrite verbose;
debug level=3;
headers "Authorization"="Bearer &ACCESS_TOKEN" "JSON-Parameters"='{ "casout": { "caslib": "public", "name":"class","promote":"true" },"importOptions": {"fileType":"basesas"} }';
run;quit;
Verify the results
Check the GUI to see if the data is available:
Conclusion
Once more we showed how we can load data from a SAS94 client to the SAS Viya CAS engine. That last one created by an Azure Quickstart template. Instead of modifying the initial Azure Quickstart architecture or adding an extra Azure Load Balancer to the architecture we now used the existing Azure HTTP Application Gateway and make use of the fact that CAS can be accessed through REST API calls. It comes down in configuring the Application Gateway with the correct Backend pools and Listener.
