SAS Workload Orchestrator Options for Availability and Encryption
SAS Workload Orchestrator is the new grid provider that ships with SAS Grid Manager 9.4 M6. It presents HTTP RESTful interfaces for ongoing operations and administration. Out of the box, the configuration of this HTTP traffic is somewhat rudimentary, with only plain-text communication (i.e. no encryption) and basic redirection provided from the master candidate hosts to the active master (i.e. limited automatic client discovery of SWO master failover).
Out of the box
When SAS Workload Orchestrator is installed and configured by the SAS Deployment Wizard, the minimum failover capability for the discovery of the current SWO master is enabled.
Default, out-of-the-box SAS Workload Orchestrator
In a SAS Grid Manager deployment, there are basically two kinds of grid server hosts:
- Grid master candidates
- Grid worker nodes
Grid worker nodes only perform the work they're assigned. A grid master directs all work on the grid - and it can also perform work as well, if desired. If there's only one host capable of acting as the grid master, then that's a single point of failure - which is antithetical to what we want from a grid. So for SAS Grid Manager, we can identify multiple hosts which have the ability to take on the role of the grid master - and they're called master candidates. The master candidates always know which host among them is currently acting as the grid master.
If an HTTP request for the current grid master is inadvertently sent to a grid master candidate, then the candidate will respond with a 300-level HTTP code which re-directs the client to make the request of the current master. This is a nifty trick, but it's not really enough to ensure high availability. The client must know which host(s) are master candidates. If there's a hardware failure, then the SWO master candidate on that host cannot reply with an HTTP 300-level code (or anything else) because it's offline.
Provide automatic client discovery of SWO master failover
Instead of requiring the grid client to keep track of all SWO master candidate hosts and then possibly attempt to contact them each one-by-one in a failure situation until it finds the current master, we can configure the SAS Web Server to act as a proxy. We will configure it to keep track of everything automatically, and then just use it as a single point of contact.
SAS Workload Orchestrator with automatic client discovery
For details on this, see my post: New Considerations for SAS Grid Manager 9.4 M6
Provide TLS encryption of SWO communication over HTTP
Have you noticed that so far all communication with the SWO has been shown as plain, unencrypted HTTP? That's not what our customer IT administration teams want to see for the ongoing operations of their critical grid infrastructure. They prefer that communication to be fully encrypted with TLS.
Encrypting SAS Workload Orchestrator communications
My GEL team comrade Ajmal Farzam explains how to enable TLS encryption for HTTPS communication of SWO operations in his blog post, SSL Encryption for SAS Grid Manager. Notice that automatic client discovery of the current SWO master is still enabled as well.
Fully realize failover and encryption
SAS Grid Manager environments often represent a significant investment in time, effort, and money by our customers. Ensuring the availability of that investment is a high priority and so all single points of failure must be addressed. Having just one SAS Web Server to act as a proxy between the grid clients and the SWO master candidates isn't sufficient - we need two of them plus another third-party load-balancer in front of them. And of course, hardening HTTP communication with TLS encryption is absolutely necessary.
Highly available and fully encrypted operation of SAS Workload Orchestrator
SAS Technical Consultant Wade Adkins authored a post to explain how to accomplish this: High Availability and TLS with SAS Workload Orchestrator.
An exercise for the reader
We use the SAS Web Server to act as a proxy in front of the SWO master candidates for one reason: because it's there. But we don't have to. If your customer site enforces a strict separation of web tier from the compute tier, then they might prefer a different approach.
Bypassing the SAS Web Server
The service labeled as "VIP Load Balancer" in the illustrations above is an oblique reference to some piece of kit that your customer must provide, configure, and maintain. It might be hardware. Or software. Or some combination. In any case, it should be implemented as a highly available service, too. Given the examples above explaining how to configure the SAS Web Server (effectively the same as Apache HTTP Server), then your customer's IT team should have sufficient information to proceed.
Coda
The SAS Workload Orchestrator is delivered with a basic configuration out of the box. Sites running production environments SAS Grid Manager 9.4 M6 will usually require configuration to improve both high availability and encrypted communication. In addition to the SAS documentation, refer to the blog posts shown here to address the customer's requirements for availability and encryption of SWO operations.
