High Availability with SAS Viya 3.4: Front-end Load-Balancing Considerations
Configuring many of the SAS Viya 3.4 components for High Availability is as easy as specifying multiple hosts in the inventory file used for deployment. This is true also for the Apache HTTP Server that sits in front of the middle tier, acting as a front door to all web applications and microservices. How do all of the other components integrate with multiple front doors?
A clustered middle tier
Previous articles describe how service discovery and routing works and how to configure a reverse proxy in front of web applications.
These posts describe how most of the automation - or "magic", as you prefer - happens thanks to the sas-viya-httpproxy service and its automatic registration of internal and external routes.
In the use case of middle-tier clustering, each Apache HTTP Server has a corresponding sas-viya-httpproxy instance, which registers its own address for both the external route (named viya) and the internal route (named httpd).
Select any image to see a larger version.
Mobile users: To view the images, select the "Full" version at the bottom of the page.
The result is that there are multiple service registrations, one per Apache node; the SAS Configuration Server monitors all of them and tracks when an Apache HTTP Server goes down. Microservices handle load balancing and high availability through a clustered middle-tier automatically; they include a service resolver that provides them with up-to-date information about where to find each active Apache HTTP Server instance. This gives them the capability of performing both round-robin load balancing, and automatic failover in case of failure of an Apache HTTP Server instance. The previous figure represents these capabilities with the multiple blue and red lines connecting microservices and Apache HTTP Server instances.
The end users' experience
If end users are presented with the simple architecture from the previous figure, they can choose to access Viya by explicitly entering in their browser the address of either of the multiple Apache frontends. Although this works, it does not provide a pleasant experience.
A virtual address for the microservices (for example, using a front-end proxy, as shown in a previous post, or a hardware load balancer) is important for the user experience because it provides the convenience of a single bookmarked entry point for SAS Viya, as shown in the following diagram:
This configuration makes single signon and signoff among SAS Viya applications possible. SAS Logon tracks session information mapped to the hostname being used to connect to it; this allows SAS Logon to share a single session for a user among many clustered instances of itself. When you log off from SAS Logon, through any running instance, you’re logged off from SAS Viya completely: all web applications in all browser sessions (for a given browser). Conversely, if you allow users to reach SAS Logon at multiple addresses, then they will create multiple sessions, which won't benefit from the single signon and signoff capability.
Some additional configuration steps
The service resolver works extremely well for the microservices and web applications, but there are just a few components that do not automatically take advantage of it. The SAS Launcher Server, CAS, and applications that generate hard-coded links to SAS Viya reports for sharing outside SAS Viya, all remain hard-wired to the Apache HTTP Server instance running on the first machine listed in the inventory file under the [httpproxy] host group. This is illustrated in the previous figure by the dotted black and red lines.
Since a cluster of Apache HTTP Servers requires an external front-end proxy or a load balancer to route external incoming connections, you can set additional properties to identify this external single entry-point also for these components that do not use the service resolver.
First, the external proxy/load balancer should be set up to use transport layer security (TLS); you must install its certificate authorities (CA) certificate chain into the SAS Viya truststore, as described in “Add Certificates to Truststores Using Ansible (Linux Full Deployment)” in Encryption in SAS Viya 3.4: Data in Motion.
Then, each component requires additional configuration steps, described in the following paragraphs.
SAS Cloud Analytics Services (CAS) Server
The CAS server does not use the SAS Configuration Server to find an active instance of Apache HTTP Server. It stores in its configuration a URI to locate SAS Viya microservices in order to validate OAuth tokens being presented for authentication from other SAS Viya web applications or to obtain OAuth tokens for clients authenticating with user/password credentials. By default, this URI is fixed to the Apache HTTP Server that is deployed on the first machine listed for the [httpproxy] inventory host group. You can override this default by setting the value for its properties cas.SERVICESBASEURL, env.CAS_VIRTUAL_HOST, env.CAS_VIRTUAL_PORT, and env.CAS_VIRTUAL_PROTOCOL to point to the front-end external proxy/load balancer that routes traffic across multiple Apache HTTP Servers.
Example:
cas.SERVICESBASEURL = 'https://loadbalancer.viya.customer.com:443/'
env.CAS_VIRTUAL_HOST ='loadbalancer.viya.customer.com'
env.CAS_VIRTUAL_PORT = '443'
env.CAS_VIRTUAL_PROTOCOL = 'https'
If you know the correct values for these properties before the deployment, you can specify them in the vars.yml file as described in SAS Viya 3.4 for Linux: Deployment Guide. Otherwise, you can change them post-deployment, by updating the configuration file /opt/sas/viya/config/etc/cas/default/casconfig_usermods.lua, and then restarting the CAS controller.
Of course, CAS must have a network route to the load balancer.
SAS Launcher Server
When Compute Servers are launched, they ultimately need to talk back to the Compute service and should do so through the load balancer. By default, however, the Launcher Server passes only the address of the first Apache HTTP Server instance listed in the inventory file to each launched Compute Server. The SAS Launcher Server requires a property retrieved from the SAS Configuration Server (config/launcher-server/global/sas-services-url) to specify the URL for the single entry point to SAS Viya. If you know this URL before the deployment, you can specify the required property in the sitedefault.yml, following the instructions in SAS Viya 3.4 for Linux: Deployment Guide. Otherwise, you can add the property post-deployment using the following commands:
. /opt/sas/viya/config/consul.conf
export CONSUL_TOKEN=$(sudo cat /opt/sas/viya/config/etc/SASSecurityCertificateFramework/tokens/consul/default/client.token)
/opt/sas/viya/home/bin/sas-bootstrap-config kv write config/launcher-server/global/sas-services-url "https://loadbalancer.viya.customer.com:443/"
In this latter case, restart the sas-viya-runlauncher-default service.
Again, a network path must exist from the Compute Server hosts to the load balancer and not be preempted by a firewall rule.
URL Generators
Applications that generate links to SAS Viya objects, such as SAS® Visual Analytics reports, also require a configuration change, specific to the SAS Viya service, that will override an otherwise hard-wired affinity to the primary Apache HTTP Server instance. You can use the properties config/viya/sas.httpproxy.external.hostname and config/viya/sas.httpproxy.external.port. Again, if you know these properties before the deployment, you can specify them in the sitedefault.yml, following the instructions in SAS Viya 3.4 for Linux: Deployment Guide. Otherwise, you can add them post-deployment using the following commands:
. /opt/sas/viya/config/consul.conf
export CONSUL_TOKEN=$(sudo cat /opt/sas/viya/config/etc/SASSecurityCertificateFramework/tokens/consul/default/client.token)
/opt/sas/viya/home/bin/sas-bootstrap-config kv write config/viya/sas.httpproxy.external.hostname <loadbalancer_hostname>
/opt/sas/viya/home/bin/sas-bootstrap-config kv write config/viya/sas.httpproxy.external.port <loadbalancer_port>
After this, restart the sas-viya-reportdistribution-default and sas-viya-reportalerts-default services.
Conclusion
This post describes some basic deployment architectures. Additional configuration steps may be required for different environment designs, such as:
- Firewalls that prevent internal components from connecting to the external proxy/load balancer
- Authentication schemes such as SAML or OAuth 2.0/OpenID Connect
- Multi-tenancy.
