SAS Viya 2020.1 (and later) Full-Stack TLS with Cert-Manager Ingress Certificate
In this post we will continue our series looking at encryption of data in motion with SAS Viya 2020.1 (and later). In this post we’ll look at taking the secure by default configuration we reviewed before and extending it. The extension we will examine is using cert-manager to generate the Ingress Controller certificate pair (private key and signed server identity certificate). However, this time we will use a different cert-manager issuer for the Ingress Controller (NGINX) rather than the default sas-viya-issuer.
Why?
I believe that this scenario will be popular. It enables customers to use any of the supported cert-manager issuers to generate the certificate pair (private key and signed server identity certificate) for NGINX. In this post we’ll look at using a simple example issuer. But the same steps could be taken to use an ACME issuer and a CA like Let’s Encrypt to generate the Ingress Controller certificate pair (private key and signed server identity certificate).
Requirements of Let’s Encrypt
Remember to use Let’s Encrypt your Kubernetes environment must be accessible by Let’s Encrypt. For the ACME CA server to verify that a client owns the domain, or domains, a certificate is being requested for, the client must complete "challenges". This is to ensure clients are unable to request certificates for domains they do not own and as a result, fraudulently impersonate another’s site. As detailed in the RFC8555, cert-manager offers two challenge validations - HTTP01 and DNS01 challenges.
HTTP01 challenges are completed by presenting a computed key, that should be present at a HTTP URL endpoint and is routable over the internet. This URL will use the domain name requested for the certificate. Once the ACME server is able to get this key from this URL over the internet, the ACME server can validate you are the owner of this domain. When a HTTP01 challenge is created, cert-manager will automatically configure your cluster ingress to route traffic for this URL to a small web server that presents this key.
DNS01 challenges are completed by providing a computed key that is present at a DNS TXT record. Once this TXT record has been propagated across the internet, the ACME server can successfully retrieve this key via a DNS lookup and can validate that the client owns the domain for the requested certificate. With the correct permissions, cert-manager will automatically present this TXT record for your given DNS provider.
Setup Issuer
The first step is to setup the additional issuer which will be used for the Ingress Controller (NGINX). For example, we can create the following YAML file for a simple CA issuer:
NS={{yournamesapce}}
tee ~/project/deploy/${NS}/site-config/security/cert-manager-ca.yaml > /dev/null <<EOF
---
# sas-viya-ingres-ca-certificate.yaml
# ............................
apiVersion: v1
kind: Secret
metadata:
name: sas-viya-ingress-ca-certificate-secret
data:
ca.crt: $BASE64_CA
tls.crt: $BASE64_CERT
tls.key: $BASE64_KEY
---
# sas-viya-ingress-issuer.yaml
# ....................
apiVersion: cert-manager.io/v1alpha2
kind: Issuer
metadata:
name: sas-viya-ingress-issuer
spec:
ca:
secretName: sas-viya-ingress-ca-certificate-secret
EOF
In our case this will create a Kubernetes secret containing a root CA certificate, and an Intermediate CA certificate pair (private key and signed certificate). Then it creates the CA cert-manager issuer using that Kubernetes secret.
These two items will need to be created in your SAS Viya namespace before deploying SAS Viya. You can apply this YAML to Kubernetes with the following:
NS={{yournamesapce}}
kubectl -n ${NS} apply -f ~/project/deploy/${NS}/site-config/security/cert-manager-ca.yaml
You can confirm the new issuer is created correctly using the following command:
NS={{yournamesapce}}
kubectl -n ${NS} get issuers sas-viya-ingress-issuer -o wide
You should see the following:
NAME READY STATUS AGE
sas-viya-ingress-issuer True Signing CA verified 19m
This completes setting up our sample issuer for the Ingress Controller.
SAS Viya kustomization.yaml
As we’ve discussed before the secure by default configuration for SAS Viya 2020.1(and later) is driven by the kusomization.yaml file. All of the standard documented entries are placed in the kustomization.yml.
The difference this time will be in the single file, which is customized for the customer’s specific environment, the site-config/security/cert-manager-provided-ingress-certificate.yaml. This file now references the new issuer we have created. For example:
---
apiVersion: builtin
kind: PatchTransformer
metadata:
name: sas-cert-manager-ingress-annotation-transformer
patch: |-
- op: add
path: /metadata/annotations/cert-manager.io~1issuer
value: sas-viya-ingress-issuer
target:
kind: Ingress
name: .*
You can then build the updated site.yaml and deploy SAS Viya as per the documentation:
NS={{yournamespace}}
kustomize build -o site.yaml
kubectl apply -n ${NS} --selector="sas.com/admin=cluster-wide" -f site.yaml
kubectl wait -n ${NS} --for condition=established --timeout=60s -l "sas.com/admin=cluster-wide" crd
kubectl apply -n ${NS} --selector="sas.com/admin=cluster-local" -f site.yaml --prune
kubectl apply -n ${NS} --selector="sas.com/admin=namespace" -f site.yaml --pruneResults
Once the site.yaml has been applied the Ingress Controller (NGINX) will now request a certificate pair (private key and signed server identity certificate) from the new sas-viya-ingress-issuer. This will only impact the HTTPS connections into the environment. When the Ingress Controller re-encrypts traffic to the SAS services this will be using the certificate pairs issued to the services by the default sas-viya-issuer we discussed before.
However, it is useful to understand what actually happens when the site.yaml is applied. The following diagrams illustrates this:
Select any image to see a larger version.
Mobile users: If you do not see this image, scroll to the bottom of the page and select the "Full" version of this post.
The first thing to notice is that we now how two different kubectl commands creating items within the SAS Viya namespace. The items shaded in green are the sas-viya-ingress-issuer, and in our example the associated intermediate CA. Which were created before deploying SAS Viya.
When we deploy SAS Viya, we see the following:
- As before we still have the internal sas-viya-ca-certificate-secret, generated by the sas-viya-selfsigning-issuer, and the sas-viya-issuer
- The sas-viya-issuer is used by the sas-certframe init container to request the certificate pairs for the individual SAS services
- The sas-viya-issuer is used to generate all the SAS service certificate objects and the associated Kubernetes secret holds the certificate pair
- The Kubernetes Ingress definitions for each SAS service now reference the separately defined sas-viya-ingress-issuer
- The updates to the Ingress definitions trigger an update of the nginx.conf to include the private key and certificate information from the sas-ingress-certificate Kubernetes secret
- The Kubernetes secret sas-ingress-certificate holds the certificate pair for the Ingress Controller and the associated Kubernetes secret leverages the sas-viya-ingress-issuer
- A Kubernetes configMap is created referencing the name of the Kubernetes secret holding the Ingress Controller certificate
- A Kubernetes configMap is created defining the location of the truststores for use by the different SAS services
This results in the Ingress Controller using the certificate pair generated by the sas-viya-ingress-issuer, while all the SAS services use their own certificate pairs generated by the sas-viya-issuer. HTTPS traffic from outside the Kubernetes cluster will be encrypted by the Ingress Controller and then re-encrypted, using a different certificate pair, for the internal network communication. However, all services are fully configured to use TLS, it is just the certificate pairs used by different components that change.
Closing Remarks
In this post we have shown how easy it is to keep SAS Viya secure by default, but quickly replace the Ingress Controller certificate pair. This means that browser clients connecting to SAS Viya will only be concerned with the Ingress Controller certificate pair. So, our end-users will only see this certificate in their browsers. Which, so long as it is signed by a CA the browser trusts, means there will be no further client configuration to complete.
In our example above we have used an internal Intermediate CA to setup the sas-viya-ingress-issuer. If the customers environment can be directly accessed by the Let’s Encrypt CA; then the same scenario can be used to issue the Ingress Controller certificate pair by Let’s Encrypt. Using Let’s Encrypt means that customers will not need to worry about the CA being trusted by most browsers and Let’s Encrypt will also handle renewing the certificate pair for them.
You might ask "why can’t I use Let’s Encrypt for all certificate pairs?". Remember from the requirements for Let’s Encrypt covered above the external Let’s Encrypt need to prove that you "own" the domain name used in the common name of the certificate. For all the internal SAS Viya services this common name is based on the pod name. For example, an instance of SAS Logon Manager the common name would look like sas-logon-app-6fc9f6696f-jrl75. Let’s Encrypt will treat this like a network domain name and from outside your Kubernetes cluster attempt to resolve it. Such an external DNS resolution will fail and so Let’s Encrypt will be unable to issue the certificate pair.
Note: This article addresses features of SAS Viya 2020.1 (and later).
