Services not restarting after OS patching

[Erict]

Hi, over the weekend we did some OS security updates where our Viya install resides (linux environment).

After patching was complete, the services did not fully restart.

Basic troubleshooting of bringing the services down and back up led to a majority of them being "down" or "not ready", so we rolled back to a VSphere snapshot taken about 16~ hours before (prior to any changes taking place). 

Reviewing logs, found some certificate errors.

2024-06-16 20:48:49.144 ERROR 16999 --- [ main] c.s.c.rest.boot.vault.CertificateUtil : service [VAULT_CERTIFICATE_REQUEST_ERROR] Vault PKI back end failed to issue certificate.

2024-06-16 20:47:29.064 INFO 16999 --- [ main] c.s.c.rest.boot.vault.CertificateUtil : service [VAULT_CERTIFICATE_REQUEST] Requesting SSL certificate from Vault PKI back end for: cawina06.cyphersystems.com

2024-06-16 20:47:29.079 WARN 16999 --- [ main] c.s.c.rest.boot.vault.CertificateUtil : service Encountered exception issuing certificate from Vault. 

org.springframework.vault.VaultException: Status 400: cannot satisfy request, as TTL is beyond the expiration of the CA certificate

We are considering running the playbook renew-security-artifacts.yml but we are unsure of what the side effects of this would be and if we could make matters worse doing so.

-Eric

[gwootton]

I think you've correctly identified the issue and solution. The SAS Secrets Manager (Vault) process has a CA with a certificate that expires earlier than the TTL for the new certificate it's trying to generate. The "renew-security-artifacts.yml" playbook should re-issue that CA certificate.

Renew Security Objects Using Ansible Plays (Linux Deployment)
https://go.documentation.sas.com/doc/en/calcdc/3.5/calencryptmotion/n1xdqv1sezyrahn17erzcunxwix9.htm...