Backup Strategy Recommendations for SAS Viya on Kubernetes (On-Premises)

[Mahis]

Hi Community,

We have a SAS Viya 4 environment deployed on Kubernetes. Our infrastructure is entirely on premises and we're looking to implement a robust internal and external backup strategy.

Could you please advise on the specific components we should back up and the recommended frequency for each?

Thank you.

[Sajid01]

Hello 
Kubernetes is a container orchestration tool.  what is your deployment platform?

[Mahis]

Red Hat 8.8

[Sajid01]

Hello @Mahis 
Typically, on-premise servers are VM's and these  are regularly backed by the infrastructure /storage team.

In case of an issue the entire VM can be restored.
In my experience with SAS 9.4, if there is a major change, snapshot/backup of the VM is taken and in case of an issue, it is restored. I know   for DR testing SAS grid is restored from the backups and works as expected.
You can draw a plan in consultation with infrastructure/ Linux Admin/storage teams.

[gwootton]

Viya has backup functionality that makes use of persistent volume claims to store backups.
If you are using your own on-prem cluster, you would want to have some sort of backup solution of the storage volumes that underpin the persistent volume claims, as well as backups for anything not backed up by the Viya backup functionality.

More information on what the Viya backup covers can be found here:

https://go.documentation.sas.com/doc/en/sasadmincdc/v_057/calbr/p1hkatn0s9p84fn19swyxm88fors.htm#n1n...

[Mahis]

What are the common backup solutions for this on an on-prem Red Hat cluster?

[SASKiwi]

Your IT storage experts should be able to provide suitable advice on this. Using the storage tools already available in your server infrastructure is the best approach.

[Mahis]

What things or file systems should we back up externally?

[SASKiwi]

Usually companies have default backup procedures for all permanent on-premises storage. My company does. Find out from your storage experts what they are - daily, weekly, monthly, incremental or full. The key thing is to ensure all SAS server permanent storage is backed up, including SAS applications and the data for these. Exclude SAS temporary storage from being backed up.

You should really have a discussion with your management regarding backup policy. There may already be backup standards in your company that you need to adhere to. That's certainly the case with mine and we walked through the requirements at the server design stage.

[JuanS_OCS · Accepted solution]

Hi,

I think you received good replies here.
Indeed, your IT probably has a backup strategy that any part of your infrastructure will follow suit. However I am guessing you are part of IT and you would like to know the specifics for SAS when running on Kubernetes pods, since it’s probably different in your infrastructure.

So, yes indeed, regarding incremental and full backups, it’s probably pre-defined at storage level.

Let’s find it what you have in SAS:
- VMs K8s cluster with nodes and Persistent Volumes (local or shared storage).
- Stateful, Stateless, Compute, CAS and System nodes (at least)
- A jump server where you operate SAS Viya from, with your kustomize playbooks and deployment assets.

For the VMs and pods, normal backups, run only when a new version is deployed, is the bare minimum. Any more frequent is fine but won’t make a difference.

Stateful nodes: One key aspect was mentioned by @gwootton, SAS offers an internal backup. This one is critical, as you can do a backup of all the storage, however you need to ensure the state of the system, which is in the Stateful nodes, and will not be in storage unless you run this internal backup, as these services run in memory. If you align to run the internal backup before your PVs backup with incremental backup you are good to go.

- Stateless: nothing in particular, unless you want to keep your logs, which is a different discussion.

- Compute: similar as above: the local PVs for SASWORK or UTILLOC won’t need any backup unless really needed by users and batch processes. Of course any data related PVs and datasources (git, databases, etc) will need daily incremental backups.

- CAS: you probably want at least a daily backup of CAS_CACHE PVs. For the data backups, similar as Computer.

- System and jump server: probably daily backups are a good idea too. 😉

[Mahis]

@Sajid01 @gwootton @SASKiwi @JuanS_OCS 

Thanks a lot for all this information.