How do I kustomize a SAS Viya deployment?

5 Likes

In recent months I have been working on the enablement materials for the new release of SAS Viya. You no doubt know by now this release will only run on Kubernetes. This will mean changes for SAS Viya Administrators. One of the tools the administrator is going to have to get used to is Kustomize. In this post I will introduce kustomize and share some experiences of using this tool and SAS Viya on Kubernetes.

There is a lot to learn about Kubernetes and SAS Viya Deployment in this blog post I am going to skip the details and keep it simple with a few key facts.

A Viya deployment in Kubernetes is made up of set of Kubernetes resources which work together to run the software in a Kubernetes cluster.
To create SAS Viya on the cluster Kubernetes accepts manifests. A manifest is just one or many yaml files.
Kubernetes manifest describes the resources (e.g., Deployments, Services, Pods, etc.) you want to create, and how you want those resources to run inside a cluster.
When you apply the manifest Kubernetes creates or modifies the resources in the cluster.

When you perform a deployment SAS delivers a directory with deployment assets. The deployment assets include the following:

Select any image to see a larger version.
Mobile users: To view the images, select the "Full" version at the bottom of the page.

Using the kustomization yaml file the kustomize software reads the other yaml files, SAS provided and user created, and generates a manifest that describes the complete deployment. Applying the manifest creates your Viya deployment.

This K8S manifest describes the system and can be modified to change/update aspects of the system. In working on the SAS Viya training materials it quickly became apparent that updating the manifest file is not just a deployment time activity, it is also going to be a common task for SAS Viya administrators.

From an administration perspective the changes you might make may include:

Making external content available to SAS Viya in the cluster
Providing access to permanent storage
Allocating or restricting resources such as memory or cpu.
Starting/stopping resources

How do we make changes to the Kubernetes manifest after deployment? There are a few tools available that could help, or we could just edit it, but the tool of choice for Viya deployments is Kustomize. What is Kustomize?

Kustomize is a tool that can be used to create or update k8s manifests. Kustomize “traverses a Kubernetes manifest to add, remove or update configuration options”. Kustomize accepts a kustomization.yaml file as input and, from its content, creates/updates a Kubernetes manifest (in this post the manifest will be named site.yaml). So we have yaml, creating/updating yaml.

The overall flow of what an administrator would do is illustrated in the diagram below.

To summarize.

Create user-defined yaml files and store them in a directory (site-config) that is a sub-directory of the location where kustomization.yaml file resides
Edit the kustomization.yaml file to reference the newly created yaml file
Use kustomize build to build the Kubernetes manifest (site.yaml)
Use kubectl apply to update the environment (only the resources that have changed will be updated)

The diagram below, illustrates the process.

But what about the details? There are a number of ways that this can happen, some more complex than others. Lets first look at a simple example.

Simple Example: Add a Persistent Volume Claim

Using resources in kustomize we can define a complete Kubernetes resource definition in a new yaml file. The process will then include this resource in the generated manifest.

Step 1) Define new resource in a file migration_package_pvc.yaml.

Create a new yaml file for a persistent volume claim. In this case we include in the yaml the complete definition of a Kubernetes persistent volume claim (PVC).

Step 2) Update kustomization.yaml to reference new resource definition file.

Update the kustomization.yaml file to reference the file that defines the resource definition of the persistent volume claim. The resources section contains references to resource definitions that will be included in the resultant Kubernetes manifest.The other sections such as transformers and patches are more complex and we will look at them later.

Step 3) Use the kustomization.yaml to build the manifests (site.yaml)

In the directory where you kusomization.yaml is located execute the command: kuztomize build - o site.yaml The site.yaml manifest file that is built will now include the definition of the new resource. The yaml from site-config/migration_package_pvc.yaml is simply included in the site.yaml manifest file.

When the manifest is applied using "kubectl apply" the persistent volume claim is created in the cluster. If you are a Base SAS program this is similar to a %include in SAS code where you just include some code from another program in your current program.

More Complex Example: Adding nfs mount volume for CAS

That was easy! Lets look at a more difficult example. In this case we have an NFS share that contains the data that needs to be mounted inside the CAS pod. In this more complex example, instead of adding a complete resource, we will make changes to a resource that already exists. The resource exists in the cluster and is already defined in the manifest (site.yaml).

Step 1) Create a transformer definition yaml file.

In kustomize transformers modify resource configuration by adding, updating or deleting fields. The transformer yaml is not a complete k8s resource definition but instructions on how kustomize should add/update k8s manifest. In the transformer you specify the target to locate the exact resource or resources that you want to update. The target subsets the resources that the transformer will operate on. Kind: CASDeployments group: viya.sas.com Name: all names Version: v1alpha1

Note: this is a pretty basic way to subset, there is a lot of powerful ways to subset and identify the Kubernetes resources to operate on.

For example this one would apply to all all Deployments that have a sas.com/deployment-base label that includes spring or go.

target:

kind: Deployment

labelSelector: sas.com/deployment-base in (spring,go)

In addition to the target you specify the patch. The patch defines the :

Operation in this case add (add|update|delete)
Path traverses the yaml data structure to the exact location to add the site
Values are the name value pairs to add at that location.

This yaml adds two “patches”. A volume and a volume mount to all CASDeployments in the site.yaml kubernetes manifest file.

Finding the location in the yaml to apply the patch might be the trickiest part of the whole process. Using an editor like Visual Studio Code with a yaml plugin can help to identify the path within the target yaml where the addition should be made. In the screenshot below selecting the location where the existing volumeMounts are defined shows the yaml path in the tool: /spec/controllerTemplate/spec/containers/0/volumeMounts/-.

Step 2) Update kustomization.yaml to reference new file.

Update the kustomization.yaml file to reference the transformer file created in step 1.

Step 3) Use the kustomization.yaml to build the manifests (site.yaml)

kuztomize build - o site.yaml The mainfest (site.yaml) file will now include the volume and volumeMounts in the CASDeployment resource. The definitions are added to the target resource based on the path within the yaml.

As well as transformers you will see patches used in kustomize. The two methods can achieve the same results.

The patch method uses the patches section in the kustomization.yaml file. In the patch definition specify a “row” for each operation and include:

op: is the type of operation: add.
Path: the navigation path to the object in the manifest file (same as in the last example).
value: is the parameters specified in name value pairs.

One of the differences in the patch method is that you define the target in the kustomization.yaml . The result in the end is the same.

We will need to know yaml and kustomize fairly well to be able to administer a SAS Viya deployment on k8s. The deployment assets delivered by R&D provide a good set of examples including readme files to get you started.

This post has just scratched the surface. I am sure there are others that have more experience in this area. If you have experiences to share please feel free to comment.

For more information check out the following resources:

Search for more content from our group: SAS Global Enablement & Learning.