Navigating The Hall of Mirrors

[#1]

Using a local mirror registry to deploy and update SAS Viya is a common practice, with many organisations adopting secure, air-gapped or restricted network environments. Mirror registries offer control and security, but there are some things to consider when applying updates, especially around version management, tooling compatibility, and deployment orchestration. In this post, we’ll explore some common pitfalls and good practices when using a local mirror registry for SAS Viya deployment updates.

Keeping Everything in Sync

When using a local mirror for your SAS Viya deployment, you're responsible for mirroring required container images from cr.sas.com. The key challenge is keeping those in sync with the deployment assets for the version or release you want to update to, as well as syncing those with the correct version of sas-orchestration! If your deployment assets reference image versions that don’t exist in your mirror, or if your orchestration tool is out of sync with your assets, your deployment will fail.

Using the Right Image Versions

Each SAS Viya cadence release (e.g., 20250612.1749750470829) is tied to a specific set of container image versions. If your mirror registry is missing (or has an incorrect version of) one of these images, the deployment will hang or fail during orchestration.

There are some steps you can take to avoid this:

• Use the viya4-orders-cli to fetch deployment assets for the exact cadence release you need; for instance, the release number displayed when you query available updates using the SAS Update Checker. In a dark site, you can periodically run the viya4-orders CLI on any machine (including client workstations or jump hosts) that can connect to ses.sas.com over the internet in order to retrieve the latest assets for a given cadence version.
• Update the mirror using the downloaded deployment assets to ensure the correct image versions are pulled and pushed to the mirror.
• Validate that all referenced images exist in your local registry before deploying using the mirrormgr CLI.

Mismatched Tool Versions

Tools like sas-orchestration and viya4-orders-cli are version-sensitive. The version of sas-orchestration used for the update must match the version required by the deployment assets, to avoid errors like this:

Select any image to see a larger version.
Mobile users: To view the images, select the "Full" version at the bottom of the page.

Download the version of sas-orchestration specified in the deployment assets (in $deploy/edu/sas-bases/.orchestration/images.yaml) to avoid a version mismatch and to continue with the deployment. To do so, pull it from cr.sas.com, tag it, and then deploy/update:

cloud-user@server:~/project/deploy/edu$ cat ~/project/deploy/edu/sas-bases/examples/kubernetes-tools/README.md | grep -A13 Prerequisites
## Prerequisites

To run the sas-orchestration image, Docker must be installed.
Pull the `sas-orchestration` image:

```
docker pull cr.sas.com/viya-4-x64_oci_linux_2-docker/sas-orchestration:1.142.1-20250429.1745959628012
```

Replace 'cr.sas.com/viya-4-x64_oci_linux_2-docker/sas-orchestration:1.142.1-20250429.1745959628012' with a local tag for ease of use in the examples that will follow:

```
docker tag cr.sas.com/viya-4-x64_oci_linux_2-docker/sas-orchestration:1.142.1-20250429.1745959628012 sas-orchestration

Correctly specifying flags

When running the deployment command, you might be tempted to let the deployment tools figure out the latest available release by setting the --cadence-release flag as follows:

sas-orchestration \
deploy \
--namespace edu \
--deployment-data /license/certs.zip \
--license /license/license.jwt \
--user-content edu \
--cadence-name lts \
--cadence-version 2025.03 \
--cadence-release "" \

This won't always work in a mirrored environment. Why?

• The tool will attempt to deploy the latest available release from SAS.
• If your mirror doesn’t contain that release, the deployment will fail.

A good practice is to specify the exact cadence release you mirrored (i.e. to match your downloaded deployment assets). Don't forget to add the --image-registry flag:

sas-orchestration \
deploy \
--namespace edu \
--deployment-data /license/certs.zip \
--license /license/license.jwt \
--user-content edu \
--cadence-name lts \
--cadence-version 2025.03 \
--cadence-release 20250711.1752209235788 \
--image-registry mirror.demo.com:80

Also, specifying the exact release required in your SAS Viya Orders CLI command takes things one step further in ensuring the same release number is used consistently.

Recommended Workflow

1. Download deployment assets: viya4-orders-cli -c ~/.viya4-orders-cli.env dep $order lts 2025.03 -p ~/.
2. Download SAS Mirror Manager and mirror container images (using the new assets) to your local registry.
3. Download the matching sas-orchestration version as required by the assets.
4. Run deployment specifying the exact cadence version and release.

Final Thoughts

Using a local mirror registry for SAS Viya deployments gives you control, but it also requires some discipline. The key to success is version alignment: Match your images, tools, and assets, and avoid "" or unspecified versions. The deployment tools display log output, but the root causes of problems experienced during updates aren't always immediately obvious. It's a good idea to prepare appropriately, check pod logs in the case of issues, and validate everything after updating.

By following these practices, you’ll avoid the most common pitfalls and ensure smooth, repeatable deployments, even in air-gapped environments.

Find more articles from SAS Global Enablement and Learning here.