Troubleshooting HTTPS Certificate Issues in SAS Viya: Mismatched domain names

In our previous post, we examined the issue of untrusted certificates and the steps required to identify and resolve them in your SAS Viya platform. Now, we turn our attention to another common SSL-related problem: mismatched domain names. Mismatched domain names occur when the Common Name (CN) or Subject Alternative Names (SANs) on an SSL certificate do not align with the domain being accessed. This discrepancy can lead to browser warnings, compromised security, and interrupted service availability. In this post, we'll explore how to identify mismatched domain names, the potential impacts on your platform's security, and the steps to troubleshoot and resolve these issues effectively. Understanding how to manage domain name mismatches is crucial for maintaining a secure and trustworthy SAS Viya deployment.

Note: The $viya environment variable is used on kubectl commands that require the --namespace flag. Make sure that the environment variable is created before you use these kubectl commands.

Mismatched Domain Names (or IP)

Mismatched domain names are only relevant at the ingress level of your SAS Viya environment. This means that any discrepancies in domain names need to be addressed specifically within the configuration of your ingress controller, which is responsible for handling external traffic. This issue typically arises when using IT-provided ingress certificates. If the certificate was issued with a specific domain name but the SAS Viya platform is accessed via a different domain, a mismatch will occur, leading to browser warnings and potential security concerns. Mismatched domain names can also happen when accessing the SAS Viya platform through alternate DNS names that were not included in the SSL certificate. To avoid these issues, it’s essential to ensure that any additional DNS names used to access your platform are added to the generator (OpenSSL or Cert-Manager) when creating the ingress certificate. This ensures that the certificate covers all relevant domain names and prevents mismatches during access.

Symptoms connecting via a browser

When a mismatched domain name occurs in your SAS Viya environment, users may see browser warnings indicating that the site may not be secure, such as “Your connection is not private.” Here is an example of the Error displayed if you were using Chrome:

Select any image to see a larger version.
Mobile users: To view the images, select the "Full" version at the bottom of the page.

Symptoms connecting via SWAT

When accessing CAS on a SAS Viya platform using the SWAT package, SSL verification errors may occur if the hostname of the CAS server does not match the names listed on the SSL certificate. One such error is the requests.exceptions.SSLError, which signals that the hostname specified in the request does not match any of the valid names provided by the server's certificate. This can happen when connecting to a server via its IP address rather than a fully qualified domain name (FQDN). Here are the two specific instances of this error depending on the way you try to access your platform: If you are trying to access the REST interface via an unspecified IP:

requests.exceptions.SSLError: HTTPSConnectionPool(host='20.171.235.253', port=8777): Max retries exceeded with url: /cas/sessions (Caused by SSLError(CertificateError("hostname '20.171.235.253' doesn't match either of 'aks-cas-66761195-vmss000001', 'controller', 'controller.sas-cas-server-default', 'controller.sas-cas-server-default.gelenv', 'controller.sas-cas-server-default.gelenv.svc.cluster.local', 'fradae-p02059-rg.gelenable.sas.com', 'localhost', 'sas-cas-server-default', 'sas-cas-server-default-client', 'sas-cas-server-default-controller', '10.0.106.46', '10.244.3.15', '127.0.0.1', '192.168.0.5'",),))

If you are trying to access the binary interface via an unspecified IP:

ERROR: The TCP/IP negClientSSL support routine failed with status 807ff019, hostname 20.171.235.227 port 5570
ERROR: SSL Error: Invalid subject name in partner's certificate. Subject name must match machine name.
ERROR: Failed to connect to host '20.171.235.227', port 5570.

Technical Diagnostic & Confirmation

Run these commands from a linux server that does have access to your Kubernetes cluster. It must have the kubectl and the openssl command line tools installed.

INGRESS_HOST=$(kubectl -n $viya get $(kubectl -n $viya get configmap --output name | grep ingress-input) -o jsonpath='{.data.INGRESS_HOST}'); \
echo X | openssl s_client -connect ${INGRESS_HOST}:443 -servername $INGRESS_HOST 2>/dev/null | openssl verify -verify_hostname

In this case, you should see the following output:

CN = controller.sas-cas-server-default.gelenv.svc.cluster.local
error 62 at 0 depth lookup: Hostname mismatch
error stdin: verification failed

Resolution

The steps to correct the ingress certificate depend on how it was provisioned:

If you were using an IT provided certificate and key file for your ingress certificate:

Obtain the new ingress server certificate, ingress server key, and ingress server CA certificate containing the corrected Common Name (CN) or Subject Alternative Names (SANs)
Change the files in the site-config location that is referenced in the customer-provided-ingress-certificate.yaml file in the kustomization.yaml file.
Redeploy the SAS Viya platform software to apply the changes. (be careful if you applied manual customizations after your last deployment, these will be reverted)

If you were using the SAS provided OpenSSL Certificate Generator for the Ingress Certificate:

Change the file under the site-config folder and called customer-provided-merge-sas-certframe-configmap.yaml and more specifically the SAS_CERTIFICATE_ADDITIONAL_SAN_DNS={{ ADDITIONAL_SAN_DNS_ENTRIES }} See details in SAS® Help Center - Add the External Host Name or IP Address to the SAN in the Signed Certificate
Delete the kubernetes secret holding the expired certificate
```
kubectl -n $viya delete secret sas-ingress-certificate
```

Delete the job “sas-create-openssl-ingress-certificate” (if still exists)

kubectl -n $viya delete job sas-create-openssl-ingress-certificate --ignore-not-found

Re-run the job (using yq) to recreate the Ingress certificate (minimum required)

cd ${deploy}
yq e '. | select(.kind == "Job" and .metadata.name =="sas-create-openssl-ingress-certificate")' site.yaml | kubectl -n $viya apply -f -

Or reapply you site.yaml manifest (be careful if you applied manual customizations after your last deployment, these will be reverted)

If you were using the Cert Manager Certificate Generator for the Ingress Certificate, you will need to look at the Cert Manager documentation. The option to look for is named: dnsNames. It could be implemented at the certificate level or at the CA level.

Conclusion

In this post, we explored the critical issue of mismatched domain names in your SAS Viya environment and how they can impact secure communication. Recognizing the symptoms, such as browser warnings and accessibility issues, is the first step in addressing these mismatches effectively. By ensuring that all domain names are properly configured and included in your SSL/TLS certificates, you can maintain a secure and seamless user experience. Stay tuned for our next blog post, where we will discuss self-signed certificates, what they are, their implications for security, and how to manage them effectively within your SAS Viya platform.

For a deeper understanding, I highly recommend the comprehensive 'Advanced Topics in Encryption on SAS Viya' training. This course provides a thorough exploration of encryption, covering everything from core concepts to advanced techniques. It also includes hands-on exercises and an associated lab environment, allowing you to apply what you've learned in real-world scenarios and further strengthen your knowledge of secure communication within SAS Viya.

Be sure to check out the complete series on Troubleshooting HTTPS Certificate Issues for your SAS Viya platform to guide you through various common challenges and their resolutions.

Find more articles from SAS Global Enablement and Learning here.