The SAS Workload Orchestrator is the new SAS-developed grid provider for the SAS Grid Manager offering in SAS 9.4 M6, replacing the Platform Suite for SAS product, which was the default option for grid prior to M6. SAS Grid Manager comes with some rich new features including, among numerous other changes, the option to secure grid traffic with encryption using SSL/TLS. In this post, we'll take a look at the options available for protecting communications in SAS Grid Manager at M6, and walk through the process of enabling SSL encryption in a grid environment.
Although disabled by default, all communication between grid processes can be encrypted in SAS Grid Manager. This is thanks to the new grid provider being entirely based on REST, as a result of efforts from R&D to align with the approach adopted for SAS Viya. As such, the REST-ful API calls used by the grid can be protected using SSL. This will encrypt data in transit, including the connection to the SWO web interface, which runs as a web application inside the SWO process on every grid master candidate host.
Inter-daemon communication protection is offered out-of-the-box by HMAC, which is a hashing function that in the case of SAS Grid Manager, uses the grid passphrase to hash messages exchanged between grid processes. This provides verification to a grid daemon that the REST API calls it receives are from another valid grid daemon.
The installation of SAS Grid Manager in SAS 9.4 M6 is driven entirely by the SAS Deployment Wizard, and as such, the decision to configure the grid for SSL encryption can be made at deployment time. The SDW will prompt for configuration parameters, including the noSSL option. Note that the default value is 1, which equates to a value of 'true' for noSSL. If performing a new deployment, change the value of this option to 0 as shown below.
Select any image to see a larger version.
Mobile users: To view the images, select the "Full" version at the bottom of the page.
For environments that have already been deployed, the value of the noSSL option must be changed to 0 manually in the configuration. The easiest way to do this is using the SWO web interface. Log on to http://[SWO-master]:8901/sasgrid/index.html and log on as a grid administrator. On the Configuration page, a checkbox determines whether SSL is disabled. Uncheck the box and hit the Save button.
Note that although most grid configuration changes can be performed dynamically, the grid services do need to be restarted if enabling SSL. Stop the grid by running /opt/sas/sasconfig/Lev1/Grid/gridStop.py on your primary master host.
The next step is to create a private key and certificate. At a customer site, this would typically be a task for the customer to perform. In my demo environment, I created my own private key and self-signed certificate with OpenSSL. When generating the certificate, there are some things to consider. Since the SWO process (including the web interface) runs on multiple grid master candidate hosts, the certificate should be created with wildcard notation to extend SSL encryption to all SWO masters. For example, in my demo environment, I have four servers; sasserver01.race.sas.com, sasserver03.race.sas.com and sasserver04.race.sas.com are the master candidate hosts, and sasserver02.race.sas.com is the middle tier. For this environment, the Common Name (CN) in the certificate can be specified as *.race.sas.com to accommodate all the grid servers. Additionally, Subject Alternate Names (SAN) can be specified when creating the certificate, so that all SWO master candidate hostnames are included in and protected by the one certificate. Your browser may display a warning if the certificate does not contain appropriate SANs.
In continuing with the RACE environment example, the certificate (which is self-signed in this case) should look something like the below when the contents are viewed with OpenSSL:
openssl x509 -text -noout -in sasserver.crt
Certificate:
Data:
Version: 3 (0x2)
Serial Number:
99:eb:0c:9d:a3:9e:01:99
Signature Algorithm: sha256WithRSAEncryption
Issuer: C=AU, ST=Victoria, L=Melbourne, O=SAS Institute Inc, OU=GEL, CN=*.race.sas.com/emailAddress=none@sas.com
Validity
Not Before: May 16 06:28:57 2019 GMT
Not After : May 15 06:28:57 2021 GMT
Subject: C=AU, ST=Victoria, L=Melbourne, O=SAS Institute Inc, OU=GEL, CN=*.race.sas.com/emailAddress=none@sas.com
Subject Public Key Info:
Public Key Algorithm: rsaEncryption
Public-Key: (1024 bit)
Modulus:
00:b1:cd:cf:f6:76:55:d7:d5:70:10:c1:8e:f9:bb:
20:52:58:ac:d3:1e:cc:01:08:18:44:a1:a2:c9:6e:
85:0c:d0:f6:cc:ea:c2:35:02:6c:5f:b6:e8:08:7c:
79:cb:07:3d:84:67:b5:8b:80:32:08:28:a9:02:ee:
5f:2a:0a:73:22:37:b4:7c:61:dd:04:52:0b:b4:b4:
ed:77:59:b2:87:d6:39:6c:93:87:20:14:5a:39:c1:
48:74:c0:21:82:cc:27:d3:c2:c7:83:ba:0c:13:4a:
ad:a6:c9:2f:89:37:83:87:56:3b:a2:f1:7a:8a:31:
96:b4:cd:e8:10:dd:e7:d4:e9
Exponent: 65537 (0x10001)
X509v3 extensions:
X509v3 Subject Alternative Name:
DNS:sasserver01.race.sas.com, DNS:sasserver03.race.sas.com, DNS:sasserver04.race.sas.com
X509v3 Subject Key Identifier:
69:EE:1C:C3:66:89:39:5B:EE:E5:C6:23:12:8A:F2:21:EB:2F:FC:08
X509v3 Authority Key Identifier:
keyid:69:EE:1C:C3:66:89:39:5B:EE:E5:C6:23:12:8A:F2:21:EB:2F:FC:08
X509v3 Basic Constraints:
CA:TRUE
Signature Algorithm: sha256WithRSAEncryption
06:ef:25:cf:7d:7c:40:6d:30:68:35:a5:15:78:36:8d:54:65:
5a:1c:7e:57:be:54:45:bb:6e:e5:3b:25:9a:cc:29:79:f3:c1:
1d:ac:a2:f7:7f:29:96:3c:1d:db:41:24:63:c1:79:bd:97:bc:
3f:f0:a6:9e:01:51:ed:be:32:1a:af:0a:fb:29:3f:d4:f2:90:
1e:23:4f:0e:7e:6e:4e:19:af:ee:8f:e5:f0:9b:57:b1:a8:8c:
6c:53:20:61:1c:e5:67:7a:e0:7e:dc:cd:19:d3:09:db:8d:32:
32:5a:09:42:80:11:26:ba:01:ee:4e:99:2c:77:21:62:a2:2a:
8d:44
Note the value of the DNS option; all the master candidate hosts are listed.
Once the certificate is created, it needs to be installed in the browser on the client machine and imported into the trusted CA bundle using SAS Deployment Manager on the servers as well as on client machines. The paths to the key and certificate now need to be set in the configuration so that SWO knows what to use to encrypt/decrypt messages.
There are some additional configuration parameters that also need to be supplied to the SWO so that it knows where the SSL certificate and private key are. The following options need to be uncommented in /opt/sas/sasconfig/Lev1/Grid/sgmg_usermods.sh and appropriate values supplied:
SSLCALISTLOC="/opt/sas/sashome/SASSecurityCertificateFramework/1.1/cacerts/trustedcerts.pem"
export SSLCALISTLOC
SSLCERTLOC="<path>/sasserver.crt"
export SSLCERTLOC
SSLPVTKEYLOC="<path>/sasserver.key"
export SSLPVTKEYLOC
SSLPVTKEYPASS="<server_private_key_password_if_used>"
export SSLPVTKEYPASS
There are also some additional operating-system specific options for Windows and UNIX environments. Review the documentation to see what might be required for your environment.
Once those parameters have been correctly set, the grid services can be started again using either /opt/sas/sasconfig/Lev1/Grid/gridStart.py, or by running /opt/sas/sasconfig/Lev1/Grid/sgmg.sh start on each grid host. Each host reads the certificate and key (which works thanks to our wildcard certificate) and will encrypt all traffic. The web interface, which runs inside the SWO process, is now accessible via HTTPS (i.e. https://[SWO-master]:8901/sasgrid/index.html). A padlock icon should display in the address bar indicating a secure connection as shown below.
Rob Collum wrote an article about configuring the SAS Web Server to act as a reverse proxy for grid master candidate host failover. If your environment has been set up in this manner, some additional changes are needed to allow the Web Server to successfully load-balance to the new SSL-enabled SAS Workload Orchestrator master candidate hosts. First, the SAS Web Server needs to be stopped. The following changes then need to be made in the VirtualHost definition in /opt/sas/sasconfig/Lev1/Web/WebServer/conf/swo.conf:
SSLProxyEngine on
SSLProxyVerify none
SSLProxyCheckPeerCN off
SSLProxyCheckPeerName off
SSLProxyCheckPeerExpire off
SSLCertificateFile ssl/sasserver.crt
SSLCertificateKeyFile ssl/sasserver.key
SSLCertificateChainFile ssl/sasserver.crt
Additionally, uncomment the following line in /opt/sas/sasconfig/Lev1/Web/WebServer/conf/httpd.conf:
LoadModule ssl_module "/opt/sas/sashome/SASWebServer/9.4/httpd-2.4/modules/mod_ssl.so"
The BalancerMembers also need to be amended to refer to the new URL, which now uses HTTPS. [Update 31-Oct-2019: New hot fixes in Ship Event 19w21 have resulted in changes to the URLs listed below for the BalancerMembers to redirect to. For post-19w21 environments, change the value of hcuri= to /login/login.html]
BalancerMember https://sasserver01.race.sas.com:8901 lbset=0 hcinterval=5 hcmethod=GET hcuri=/sasgrid/index.html hcexpr=ok401
BalancerMember https://sasserver03.race.sas.com:8901 lbset=1 hcinterval=5 hcmethod=GET hcuri=/sasgrid/index.html hcexpr=ok401
BalancerMember https://sasserver04.race.sas.com:8901 lbset=2 hcinterval=5 hcmethod=GET hcuri=/sasgrid/index.html hcexpr=ok401
The Web Server can then be started again. Verify that http://[SAS-Web-Server]:8901/sasgrid/index.html is accessible.
For further information on the SWO and the SAS Grid Manager in M6, refer to the official documentation.
Thank you for reading. Please leave a comment below to ask questions or share your own experiences.
SAS Innovate 2025 is scheduled for May 6-9 in Orlando, FL. Sign up to be first to learn about the agenda and registration!
Data Literacy is for all, even absolute beginners. Jump on board with this free e-learning and boost your career prospects.