Use contexts to give users a choice of CPU and memory size: Part 1 - Launcher contexts
Giving your users different-sized compute, connect or batch pods, with more or less CPU and memory is straightforward in SAS Viya. For example you might allocate more resource in larger compute sessions for SAS Model Studio, less for sessions in SAS Job Execution Server. Or you might allocate less resource for batch pods running small batch jobs, and more for pods running large batch jobs. If you think your users won't just always use the biggest context they can, you can even give them a choice of different-sized contexts and let them choose the right-sized context for the job.
A SAS Viya Launcher context has properties you can optionally use to specify values for CPU and memory requests and limits that supersede the global default values for the same things. In this Part 1 of the article, we'll define what these properties are, and see how to adjust them for a single SAS Launcher context.
In Part 2, I'll show you how to create a corresponding set of new compute contexts which you can use to test different values with an example SAS program, to illustrate the process you would follow to find out which values suit a particular task best.
The optional properties for CPU and memory request and limit values can be found on the Advanced tab of the Edit Launcher Context dialog in SAS Environment Manager:
Select any image to see a larger version.
Mobile users: To view the images, select the "Full" version at the bottom of the page.
The screenshot above shows a launcher context with all four of these properties populated with values but you can leave some empty, and they are all empty by default. With no values set in a launcher context, SAS programming run-time pods started with the launcher context use all global default values for CPU and memory requests and limits. Each property that you populate with a value will change that property for pods launched under this launcher context.
You can also change the global defaults if you want to change these values for all launched pods except where the pod's launcher context overrides a default, and I'll touch on how at the end of part 2 of this article, but we will focus more on using context-specific values in both parts of the article.
You can see and set the launcher-specific CPU and memory values through the sas-viya command line interface (CLI) too; we will look at how to configure these properties using both the web UI and the CLI.
The net effect of all of the above is that you can 'tune' the CPU and memory requests and limits for programming run-time containers to suit the compute, or connect or batch workload they will process, and to make best use of the CPU and memory available on your compute nodes.
This gives you, the SAS Viya administrator, control over how you manage resources for compute workload and also the ability to delegate some of that control to your users where appropriate.
To properly explain different-sized launcher contexts, I've structured this two-part article in five sections:
In this part 1 - launcher contexts:
- What are CPU and memory requests and limits?
- How to configure launcher contexts with non-default values of CPU and memory request and limits
In part 2 - compute contexts and changing global defaults:
- How to configure compute contexts with SAS system option values to match the custom launcher context
- A demonstration of the impact, using some example SAS code
- How to change the global default CPU and memory request and limit
Part 1, Section 1: What are CPU and memory requests and limits?
Let's review what CPU and memory requests and limits are, because we talk about all four properties in the same breath throughout both parts of this article, and that makes some of the explanation a bit long and convoluted. Here follows my attempt at a plain-English explanation - it is not intended to be exhaustive and may contain small inaccuracies, which I will try to minimize. An official explanation can be found here: Kubernetes: Resource Management for Pods and Containers.
In his SAS Communities article Considerations for optimizing SAS compute sessions in SAS Viya on Kubernetes, Hans Joachim Edert describes how the CPU and memory requests and limits affect the performance of compute sessions in a wider context which also covers optimizing I/O throughput and pod startup latency. It's a great read, and still relevant.
And in this SAS Communities article cgroups v2 changes in Kubernetes 1.28 and why it impacts your CAS Server availability, Raphael Poumarede explains how the new linux cgroups v2 are able to kill CAS pods which use more memory than they declared they would: the same restrictions are relevant for launched compute, connect and batch pods in the SAS programming run-time.
CPU and Memory Requests
SAS Viya (and other) processes run inside a container in a Kubernetes pod, and the processes use compute resources including CPU time and memory. The request values for CPU and memory in a container are only used along with the other considerations, to decide where (and when, if the cluster is busy) the pod can be run, to help avoid running so many pods on a Kubernetes node the pods are starved of resources.
The container and the processes running inside it don't usually have any CPU time or memory which is reserved to them for their use alone, at least not until some memory is allocated to the processes, when that bit of memory is temporarily theirs and theirs alone. As they run, processes in containers must ask for CPU time, and for memory to be allocated from a shared pool of available memory on the Kubernetes node.
The way in which pods are placed on nodes depends on whether or not SAS Workload Orchestrator is enabled for workload management, or not, in which case Kubernetes handles workload management for compute, connect and batch pods just as it does for other pods.
- When SAS Workload Orchestrator is not enabled, Kubernetes decides where to start a pod. It considers several things including node labels, the pod's affinity for node labels, taints on nodes and tolerations that some pods may have for those taints, how busy each node is, and the requested amount of CPU and memory which the pod asks should be available on the node in order for the pod to be run on that node.
- When SAS Workload Orchestrator (WLO) is enabled, as it is by default in recent releases of SAS Viya, SAS WLO queues, host limits, user limits and other WLO logic are added to the above list of considerations which WLO uses to decide when and where to start a launched SAS Programming Run-time pod.
The processes running in the container don't necessarily use all of the requested CPU or memory at all times - they can be idle while they wait for some condition to be met, e.g. they might spend some time - perhaps quite a lot of their time - waiting for data to be read from or written to a network address or from storage. Kubernetes does not reserve any CPU or memory exclusively for a pod to use, thus preventing other processes from using it. However, SAS Workload Management can reserve CPU and memory for each pod it launches on a node, and make sure other pods are not started on that node so that the limit amounts of CPU and memory would not be available to the pod.
CPU and Memory Limits
Kubernetes nodes have a finite amount of CPU, sometimes expressed as a number of CPU cores because all modern CPUs are multi-core, but different CPUs have different numbers of cores. A second of CPU time is typically a second of a thread running on a core. Kubernetes doesn't allocate all of the available CPU capacity of its host node to running its workload; it reserves a bit for the OS and Kubernetes management tasks, and lets pods use the rest. Kubernetes tries to share all of the available CPU capacity among running processes in a fair way. If the total amount of CPU time that all the processes running on the node can use at any moment is less than total amount available, every process gets as much as it can use, even if that amount of CPU time is more than the self-declared CPU limit. Processes are not terminated for using more CPU time than they 'limit' themselves to. If the total amount of CPU time that all the processes running on the node can use at any moment is greater than the total amount available, their threads are allocated a portion of the available CPU time in proportion to their CPU time limit. This is meant to fairly allocate CPU time to the processes that need it most.
Kubernetes nodes similarly have a finite amount of memory and CPU. Memory and CPU are expressed in units as described in resource units in Kubernetes. Mebibytes (Mi) Gibibytes (Gi) are commonly used for memory since they are power-of-two (or perhaps powers of 1024) units, rather than the possibly more familiar but less logical Megabytes (M) and Gigabytes (G) that are based on a powers-of-ten (multiples of 1000 bytes). Fractional amounts of CPU are common in request and limit values - units of millicpu (m, one thousandth of a CPU) are commonly seen, e.g. 500m is the same as 0.5 CPU.
Kubernetes doesn't allocate the entire memory of its host node to running its workload; it reserves enough for the OS and for Kubernetes management processes, and the rest is available for pods to use. But it closely manages how much memory any pod can use; a pod which tries to allocate more memory than its self-declared memory limit is quickly terminated by a Kubernetes Out of Memory Killer (OOM killer), even if the memory on the node has not been fully allocated.
I hope my explanation above is good enough for us to be able to move on and discuss the CPU and memory requests and limits for launched compute, connect and batch pods in SAS Viya, and why you might want launcher contexts with different values for these properties, or why you might want to change the default values. Reading another explanation in the official Kubernetes documentation above, or other Kubernetes documentation about resource management may provide a description you find more intuitive.
Part 1, Section 2: How to configure and use different-sized launcher contexts in SAS Viya
Assuming you are a SAS Administrator who knows what compute, connect and batch contexts are and that they each reference a launcher context, let's walk through how to create different sized launcher contexts and use them. The process has these steps:
- If necessary and if your compute hosts have enough resource to support higher maximum values, adjust the global maximum values for CPU and memory requests and limits.
- Create a new launcher context that specifies non-default CPU and memory requests and limits. These values might be smaller or larger than those for existing contexts, but cannot be larger than the global maximum values or the compute, connect or batch session will fail to start.
- Create a new compute, connect or batch context that uses the new launcher context, and which also specifies SAS system option values which match the CPU and memory limits in the new launcher context.
The first step is only required when your new launcher context has bigger limit values than the current maximum values would permit. In general, you will always have to follow the second and third step above when making a new launcher context.
If necessary, and if you have big enough compute hosts, adjust launcher maximums
Do you intend for your new launcher context to have values for its CPU and memory requests and limits that are larger than the current maximums?
There are two places where the maximum values for CPU and memory requests and limits can be set.
In SAS Environment Manager
One is in SAS Environment Manager, on the Configuration page, All services view, filtering on 'Launcher'. Select 'Launcher service' on the left, then collapse the configuration instances on the right to find the sas.launcher.max configuration instance. See Manage Requests and Limits for CPU and Memory in the SAS Viya Administration our SAS Help Center SAS Viya Platform Administration documentation for this.
The default values for this configuration instance in my deployment, and I think in all SAS Viya deployments, are:
- cpu.limit: 2
- cpu.request: 50m
- memory.limit: 2Gi
- memory.request: 300M
See Resource units in Kubernetes for an explanation of the units used with the values above.
In a kustomize overlay
The other place where the maximum values for CPU and memory requests a limits can be specified is in a kustomize overlay referenced from your deployment's kustomization.yaml file and based on the examples launcher-memory-requests-limits.yaml and launcher-cpu-requests-limits.yaml in your SAS Viya deployment's sas-bases/examples/sas-launcher/configure directory. See Manage Requests and Limits for CPU and Memory in the Kubernetes administration part of our SAS Help Center SAS Viya Platform Operations documentation for details. (The section title is similar to the link above, but note they are aimed at different ways to change the same requests and limits).
These are not used out of the box. If you do use them, or something like them, then the max values they can specify would take precedence over the values in the sas.launcher.max configuration instance. (The default values they specify would similarly take precedence over the values in the sas.launcher.default configuration instance.)
As an aside, I think it would be confusing and a bad practice to specify your own custom values for the default and/or maximum values for CPU and memory requests and limits in both a sas-launcher kustomize overlay and in the sas.launcher.default and sas.launcher.max configuration instances. If you did, the kustomize overlays take precedence. But will you, or another administrator, remember that both exist and which takes precedence later? Pick one and document what you have done somewhere that you and other SAS and Kubernetes administrators can find in future. All else being equal, I would favor not redeploying SAS Viya just to change this value, and I would prefer to modify the values in sas.launcher.default and sas.launcher.max. Some customers may have a situation where having the Kubernetes administrator or deployment modify and control these values is preferable. Just be clear which you are using and you'll be fine.
Do you need to adjust your max resource values?
If the CPU and memory request and limit values you want your new launcher context to have are no larger than the corresponding values in the sas.launcher.max configuration instance, or the max values in the kustomization overlays if you are using them, then there is nothing to do for this step. Continue to step 2.2 below.
Are your compute hosts big enough?
If they are, you must also think about whether the larger values you want your new launcher context to have are feasible given the size of your compute node pool hosts.
How big are the hosts in terms of their available CPU and memory? What might the impact be on your existing workload if you have some compute, connect or batch pods which require or use more CPU and memory when they are started? This question is part of an architecture sizing and capacity planning discussion that should periodically take place between the SAS administrator, the Kubernetes administrator and perhaps other stakeholders in the SAS Viya environment. SAS Programming Run-time pods which use more memory or more CPU than the current ones may help performance for some workload while hindering other workload. They might not start when your deployment's compute nodes are busy, or they may hog resources and prevent other pods from starting.
If you have sufficient capacity in your nodes and you need to increase the maximum values for CPU and memory requests and limits, I would encourage you to favor doing it by editing the sas.launcher.max configuration instance.
Example Objective: we will test various sizes of launcher context with PROC MEANS
Later on in this article, I'll show how I created a base engine SAS dataset called primes, in a gel SAS library, which contains a column called random_number. The primes dataset is a surrogate for real business data; the data isn't really important for this article.
Let us suppose I want to use that dataset in a PROC MEANS statement like this:
* proc means is able to work as a multi-theaded SAS procedure;
proc means data=gel.primes fw=6 n min max median nonobs;
var random_number;
run;
I want to try running a MEANS procedure like the one above on my primes table, in compute pods with a limit of either 1, 2, 4 or 8 CPUs with the CPUCOUNT SAS system option set correspondingly and with all other settings at their default out-of-the-box values. I want to try this to see which number of CPUs is optimal for that procedure on that data, in my SAS Viya environment.
By default, the value of the CPUCOUNT SAS system option in my deployment is 4 CPUs. This means out of the box, multi-threaded SAS procedures and data step operations can use up to 4 threads for their work.
Interestingly, the initial value of the SAS Launcher cpu.limit property is 2, not 4.
As an aside, I've watched the number of threads running in a compute pod while it is running the PROC MEANS statement above, using a variety of methods which all show the same results. The compsrv process peaked at far more than four threads (like e.g. 29 or 34 total threads for the compsrv process when CPUCOUNT=4), but the peak number of threads does vary consistently up or down as the value of the CPUCOUNT SAS system option varies up or down. It looks like the relationship that CPUCOUNT is a SAS system option that limits the number of threads that multithreaded SAS procedures can use is something of an oversimplification - it's not necessarily untrue, but there's clearly more to it than that, and more threads involved in the work. It doesn't really matter though. Digging into that is well beyond the scope of this post and is not necessary for us to illustrate our point.
To enable a launcher context which specifies a CPU limit of 8 CPUs, since we are acting as SAS Viya Administrators, we will increase the cpu.limit value in the sas.launcher.max configuration instance. I changed cpu.limit to 8 in sas.launcher.max:
I changed the cpu.limit value to 8 in sas.launcher.max
In other circumstances, for other test or real use cases, you might have a need to adjust other max values in this configuration instance.
Create a new launcher context with non-default CPU and memory requests and limits
To create a new compute context which starts a SAS compute session with 8 CPUs, we will need to create a new launcher context for that compute context to reference. We can create it in SAS Environment Manager, on the Contexts page, in the Launcher context view. Right-click the SAS Studio launcher context and choose Copy from the popup context menu. Give the new launcher context a name, a description and on the Advanced tab, set CPU limit to 8. You don't have to set values for other resource properties if you are happy for the new launcher context to use the default values for those properties.
My new large launcher context sets the CPU limit to 8, but does not change any of the
other default values for CPU or memory requests or limits
Or you can create the new launcher context from the command line: first, authenticate as a user who is in the SAS Administrators custom group. Then, run something like the example command below. You can just specify one value for resource requests or limits as we did in the web interface example above, like this:
# Create a large launcher context
sas-viya launcher contexts create \
--name "SAS Studio launcher context - large" \
--description "Large launcher context for SAS Studio" \
--launch-type kubernetes \
--job-pod-template-name sas-compute-job-config \
--environment "SAS_LAUNCHER_INIT_ENCODING=utf-8;SAS_LAUNCHER_INIT_LOCALE=en_US" \
--command-allowlist "/opt/sas/viya/home/bin/compsrv_start.sh" \
--workload-orchestrator-queue "default" \
--cpu-limit 8
Or of you prefer, you could explicitly set all the resource values to the same values as the defaults except for cpu-limit, which is set to 8, like the alternative example below:
# Create a large launcher context
sas-viya launcher contexts create \
--name "SAS Studio launcher context - large" \
--description "Large launcher context for SAS Studio" \
--launch-type kubernetes \
--job-pod-template-name sas-compute-job-config \
--environment "SAS_LAUNCHER_INIT_ENCODING=utf-8;SAS_LAUNCHER_INIT_LOCALE=en_US" \
--command-allowlist "/opt/sas/viya/home/bin/compsrv_start.sh" \
--workload-orchestrator-queue "default" \
--cpu-limit 8 \
--cpu-request 50m \
--memory-limit 2Gi \
--memory-request 300M
In fact, let's create two more launcher contexts, with the following names indicating the following CPU limits (rather than the default limit of 4 CPUs). Remember to keep all launcher-specific values equal to or less than the corresponding sas.launcher.max values we set earlier:
| Launcher context name | CPU limit |
| SAS Studio launcher context - extra small | 1 |
| SAS Studio launcher context - small | 2 |
| SAS Studio launcher context - large | 8 |
Here is the code to create the other two - again I opted to set all the resource values, even the ones that are the defaults, but that's optional:
# Create an extra small launcher context
sas-viya launcher contexts create \
--name "SAS Studio launcher context - extra small" \
--description "Extra small launcher context for SAS Studio" \
--launch-type kubernetes \
--job-pod-template-name sas-compute-job-config \
--environment "SAS_LAUNCHER_INIT_ENCODING=utf-8;SAS_LAUNCHER_INIT_LOCALE=en_US" \
--command-allowlist "/opt/sas/viya/home/bin/compsrv_start.sh" \
--workload-orchestrator-queue "default" \
--cpu-limit 1 \
--cpu-request 50m \
--memory-limit 2Gi \
--memory-request 300M
# Create a small launcher context
sas-viya launcher contexts create \
--name "SAS Studio launcher context - small" \
--description "Small launcher context for SAS Studio" \
--launch-type kubernetes \
--job-pod-template-name sas-compute-job-config \
--environment "SAS_LAUNCHER_INIT_ENCODING=utf-8;SAS_LAUNCHER_INIT_LOCALE=en_US" \
--command-allowlist "/opt/sas/viya/home/bin/compsrv_start.sh" \
--workload-orchestrator-queue "default" \
--cpu-limit 2 \
--cpu-request 50m \
--memory-limit 2Gi \
--memory-request 300M
This done, you should be able to refresh the view in the Launcher contexts view on the Contexts page in SAS Environment Manager, and see the new launcher contexts:
Now that you have created custom-sized launcher contexts, you are ready to create custom compute, connect or batch contexts to use them. We will move on to that, and how to adjust the global default values for CPU and memory requests and limits, which are used wherever the launcher context does not override the defaults, in Part 2.
See you next time!
Find more articles from SAS Global Enablement and Learning here.
