SAS Viya 2020.1 (and later) Proxy Headers and OIDC
SAS Viya 2020.1 (and later) requires a supported Ingress controller. Customers can still deploy additional load-balancers or reverse proxies in front of the Ingress controller. Adding further network devices or services between the end-user and the Ingress controller can have an impact on the HTTP headers sent to SAS Viya. The HTTP headers are important as they are used at different stages of OpenID Connect authentication processing. Having incorrect headers can prevent the OpenID Connect working correctly. In this blog we will look at a simple approach to correct these headers.
Ingress Controller & Network Topology
SAS Viya 2020.1 (and later) only supports the Kubernetes NGINX Ingress Controller, as documented here. This is the version of NGINX from https://kubernetes.github.io/ingress-nginx/ and not the nginxinc/kubernetes-ingress version. You can find more details about the two different versions of the NGINX Ingress Controller here. In the basic scenario the client connects directly to the Ingress controller and is routed to the SAS Viya web applications, as shown here:
Select any image to see a larger version.
Mobile users: To view the images, select the "Full" version at the bottom of the page.
Alternatively, the customer might deploy a reverse proxy service between the client and the Ingress controller, as shown here:
This reverse proxy service could be used to load-balance request and/or as part of the network security design. So that only the reverse proxy is exposed to the network where the clients are located. Taking this a stage further a customer might deploy a specific network security device or service between the client and the reverse proxy, as shown here:
With this blog we are not attempting to recommend one topology over another. We are just illustrating that customers might place multiple devices or services between the client and the Ingress controller, and hence SAS Viya. As we add these items into the topology the HTTP(S) traffic now flows through more components and the choices for where to enable Transport Layer Security (TLS) increase.
Many customers will choose to deploy TLS on all connections in their environment. This is a common choice that often forms part of a "zero-trust" architecture and is becoming widely accepted. Other customers might choose to only deploy TLS on the outer most connection, shown as 1 in all three diagrams. These customers might "trust" the security of the rest of the connections as being "inside" their network perimeter.
The Ingress controller must have a valid DNS entry, as covered in the documentation here. Also, each of the additional network devices will most likely also have a respective DNS entry leading all the way out to the DNS name used by clients to connect to SAS Viya. In both, the second and third topologies, the DNS name used to access SAS Viya will not be the same as the DNS name for the Ingress controller. For example, in the first topology end-users might enter https://internal.viya.customer.com/SASStudio; while in the second and third topologies the URL might be https://customerportal.customer.com/SASStudio.
Proxy Headers and OIDC
When OpenID Connect is configured for SAS Viya 2020.1 (and later) SAS Logon Manager must construct the correct redirect_uri parameter to send as part of the authentication request to the OpenID Connect provider. This redirect_uri parameter is built using header values on the request to SAS Logon Manager. The headers that impact the construction of the redirect_uri are the values from the Host, X-Forwarded-Proto, and X-Forwarded-Port headers. The redirect_uri parameter must match the value or values registered with the OpenID Connect provider. It will also be used by the OpenID Connect provider to redirect the client browser back to SAS Logon Manager after authentication with the authorization code. As such, we want these header values to reflect the external address used by the end-users.
In the first topology where end-users directly access the Ingress controller this is not normally an issue. The default setup of the Kubernetes NGINX Ingress Controller will automatically set the Host header to the Ingress controller hostname and correctly set the X-Forwarded-* headers to the values for the Ingress controller. However, in both the other topologies we can run into issues with the default configuration. The Kubernetes NGINX Ingress Controller default configuration bases the X-Forwarded-* header values on the Ingress controller and overwrites any X-Forwarded-* headers sent by upstream devices or services. As we have said having the incorrect X-Forwarded-* headers can cause the OpenID Connect authentication to fail.
Changing the Default Configuration
The Kubernetes NGINX Ingress Controller has a configuration setting "use-forwarded-headers" that will change the behaviour. If "use-forwarded-headers" is false NGINX ignores incoming X-Forwarded-* headers, filling them with the request information it sees, so its own values. This is the default setting. Changing the value of "use-forwarded-headers" to true means that NGINX passes the incoming X-Forwarded-* headers to the SAS Viya web applications. As the documentation states this is the option that should be used when NGINX is behind another L7 proxy/loadbalancer that is setting the X-Forwarded-* headers. This will also set the Host header based on the X-Forwarded-Host header.
So now that we know the configuration setting that must be changed from the default value the question becomes how to apply this configuration change. The simplest approach, which works well for testing, is to update the Kubernetes configMap that is created when the Kubernetes NGINX Ingress Controller is deployed. For example, you would create a YAML file with the following contents:
apiVersion: v1
kind: ConfigMap
metadata:
name: {{HELM_RELEASE_NAME}}-ingress-nginx-controller
labels:
app.kubernetes.io/name: ingress-nginx
data:
use-forwarded-headers: "true"
This YAML file can then be applied with the following kubectl command:
kubectl -n nginx apply -f ingress_update.yaml
Finally, you can confirm the configuration option has been applied by examining the nginx.conf within one of the controller pods with the following command:
kubectl -n nginx exec -it `kubectl -n nginx get pod -l app.kubernetes.io/name=ingress-nginx -o name|tail -1` -- cat /etc/nginx/nginx.conf|grep use_forwarded_headers
You should see:
use_forwarded_headers = true,
This should be sufficient for testing. However, if the Kubernetes NGINX Ingress Controller is updated the change to the configMap could be lost.
Alternatively, this configuration option can be specified when deploying the Kubernetes NGINX Ingress Controller with HELM. This can be done either by specifying the use-forwarded-headers option in the values.yaml passed to the HELM command or by using the set statement on the HELM command. For example, to update the Kubernetes NGINX Ingress Controller with HELM and include the option in the set statements you would use the following:
helm install {{HELM_RELEASE_NAME}} --namespace nginx \
... \
... \
--set controller.config.use-forwarded-headers='true' \
... \
ingress-nginx/ingress-nginx
Then so long as the upstream device or service is correctly setting the X-Forwarded-* headers these will be available to SAS Logon Manager. Equally, if an end-user now directly accesses the Ingress controller since no upstream service has set the X-Forwarded-* headers they will be set by the Ingress controller. This is a common case when the same environment serves internal users (direct connection to the Ingress controller) and external users (connection to the frontend reverse proxy service or device).
Testing Headers
Testing the headers that are presented to the SAS Viya web applications inside Kubernetes can be difficult. The easiest way to confirm the correct headers are being passed to the SAS Viya web applications is to deploy a simple application that will echo back the request headers in a browser. For example, you can use the following YAML to define such a simple application:
apiVersion: v1
kind: Namespace
metadata:
name: echoserver
---
apiVersion: apps/v1
kind: Deployment
metadata:
name: echoserver
namespace: echoserver
spec:
replicas: 1
selector:
matchLabels:
app: echoserver
template:
metadata:
labels:
app: echoserver
spec:
containers:
- image: gcr.io/google_containers/echoserver:1.0
imagePullPolicy: Always
name: echoserver
ports:
- containerPort: 8080
---
apiVersion: v1
kind: Service
metadata:
name: echoserver
namespace: echoserver
spec:
ports:
- port: 80
targetPort: 8080
protocol: TCP
selector:
app: echoserver
---
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
name: echoserver
namespace: echoserver
annotations:
kubernetes.io/ingress.class: "nginx"
nginx.ingress.kubernetes.io/configuration-snippet: |
more_set_headers "Request-Id: $req_id";
spec:
rules:
- host: {{YOUR_INGRESS_HOST}}
http:
paths:
- path: /echo
backend:
serviceName: echoserver
servicePort: 80
You would then apply this to your Kubernetes cluster with the following command:
kubectl apply -f echo-server.yaml
This will deploy the application into its own namespace called "echoserver" and you can validate the headers sent to the SAS Viya web applications by browsing to /echo on your defined URL. For example, https://{{YOUR_INGRESS_HOST}}/echo would show the headers inserted by the Ingress controller. Equally, https://{{YOUR_REVERSE_PROXY}}/echo would show the headers available when browsing via the reverse proxy.
A Note on Security
With the Kubernetes NGINX Ingress Controller configuration, we have outlined above the Ingress controller will now accept and use any X-Forwarded-* headers it receives. This is not a problem so long as no-one else can send those X-Forwarded-* headers to our Ingress controller. Otherwise these headers could be used as an attack vector into our system.
So, we must ensure the Ingress controller is only accepting requests from our chosen reverse proxy device or service. We can do this with both firewall rules, only allowing network traffic from the reverse proxy to access the Ingress controller, and with white listing. The Kubernetes NGINX Ingress Controller can be configured with a set of whitelisted IP addresses and connections from any other IP address will just receive a 403 Access Denied HTTP response code.
Conclusion
As we have seen having the right headers is very important for the correct functioning of OpenID Connect authentication. Adding additional network devices or services into your chosen network topology can confuse the correct setting of those headers. Setting the Kubernetes NGINX Ingress Controller configuration option "use-forwarded-headers" will ensure the SAS Viya web applications receive the correct values. But, be careful to ensure only your chosen reverse proxy is then able to connect to the Ingress controller.
Find more articles from SAS Global Enablement and Learning here.
