Configure a Compute Context with Pre-started Compute Servers from the command-line

3 Likes

In this post I would like to highlight a neat way to set a useful bit of SAS Viya Programming Run-time Environment configuration from the command-line, and in the process remind you that pre-started (and reusable) compute servers are a good thing for SAS administrators to know about.

SAS Viya's Programming Run-time Environment has the ability to pre-start a pool of available compute servers. The idea is that when users request a compute session in a specific compute context in e.g. SAS Model Studio or SAS Studio, the compute session starts in a few seconds rather than a few tens of seconds because the compute session runs in a compute server that is already started. The user doesn't need to wait for the SAS Compute Service to ask SAS Launcher Service to launch a new Compute Server pod, wait for the pod to start up, and wait for all the containers in it to initialize. Those things still have to happen, but they happen before the user requests their compute session.

The apparent improvement in session start time is especially noticeable when you run something like a model pipeline which starts several compute sessions in quick succession.

The necessary steps to create a compute context which pre-starts compute servers are documented in:

However, if you like to script things so that e.g. your environment creation can be automated, read on. We're going to look at a way to enable a pool of pre-started compute servers with just three commands. Plus, the method we'll use can easily be modified to create other compute contexts with attributes and restricted access.

We have covered pre-started compute servers before. 18 months ago my GEL colleague Rob Collum wrote here about improving concurrency performance in SAS Viya. One of the things he covered was how to create a pool of reusable compute servers, and he showed how to do it using the SAS Environment Manager's web interface. And 14 months ago I used the settings we'll review today as examples in my post Add, update and remove compute context attributes with new pyviyatools. But it is a while since we've written about pre-started compute servers here, so perhaps they are worth mentioning again.

First, let's refresh our memories with a few reminders about pre-started compute contexts:

A compute context you configure to have pre-started available compute servers must run those compute servers under a shared service account
You must store credentials for that shared service account in advance using the compute service's plugin to the sas-viya CLI
- If you are running a version of SAS Viya earlier than version 2023.03 (Stable or LTS), then as Rob mentioned in his post, you cannot do this if your users identities are provided to SAS Viya by SCIM, you can only do this if they are provided by LDAP.
- However, once you have upgraded to SAS Viya version 2023.03 (stable or LTS) or later, deployments using SCIM can now also use shared service accounts to run sessions for SAS Compute Server (and other things).
Only compute servers can be pre-started, you cannot pre-start batch or connect servers
Each pre-started compute server pod consumes a bit of CPU and memory on the host nodes in your Kubernetes compute node pool, even when it is idle. Not only that, even when those pods are idle and not actually using much CPU and memory, Kubernetes effectively reserves an amount of CPU and memory for their use as specified by their container's CPU and memory 'requests' values. Having pre-started compute servers may result in Kubernetes being unable to run as many - or any - new pods on the nodes where those compute servers run, if doing so would overcommit the total CPU or memory available to Kubernetes for running pods on the node beyond the total cumulative amount of CPU or memory requested by all the containers in all the pods currently running on the node. So pre-starting compute pods effectively reserves a chunk of your total compute (or other pod-running) capacity, whether it is being used or not. TL;DR: The presence of pre-started compute server pods may prevent new compute pods from being started, even when those pre-started compute pods are idle.

But with all that taken into consideration, if your users need to run a model pipeline - i.e. a series of model steps - in SAS Model Manager, or start SAS programs in Studio, and much quicker session startup time is worth the tradeoff of running the compute session as a shared account, then creating a pool of pre-started available compute servers might be for you!

Here's how.

Step 1: Store a credential for a shared service account

You must store a credential for a shared service account whether your identities are provided by LDAP or SCIM. However, I have only had a chance to try it with LDAP so far, so the steps below are specific to shared service accounts provided by LDAP. Consult Stuart's post for how to do the equivalent when your identities are provided by SCIM.

If you are using password authentication, and your users are provided by an LDAP provider such as Microsoft Active Directory (AD) or Azure Active Directory Services (AADS), read on.

First, authenticate to the sas-viya CLI as an administrator using e.g. sas-viya auth logon, or the pyviyatool loginviauthinfo.py (described in loginviauthinfo.md).

Then optionally, run this to list existing compute credentials:

sas-viya compute credentials list

If you have not created any compute credentials yet, the response will be:

There are no shared service account credentials.

Then run this to save a compute credential for a service account called modelservice, who has password mySecretPassword123. Obviously, substitute the username and password of the service account your pre-started compute servers should actually run as for these example values:

sas-viya compute credentials create -u modelservice -p mySecretPassword123 -d "Shared service account called modelservice"

The expected response is something like:

2023/06/07 12:34:56 The shared service account credential for modelservice was created successfully.

If you like, confirm it was created by listing compute credentials again:

sas-viya compute credentials list

If the credential was created successfully, the response will be something like:

Shared Service Account Credentials:
1. modelservice - compute-password - Shared service account called modelservice

Step 2: Create JSON file that defines the new compute context

The file below is based on the example in SAS Viya Platform Administration > Servers and Services > Programming Run-Time Environment > Program.... Save it somewhere which I will refer to as /path/to/file/prestarted_reusable_modelservice_cc.json.

{
    "name": "Data Mining compute context as modelservice",
    "description": "A compute context for the SAS Visual Data Mining and Machine Learning application as modelservice.",
    "attributes": {
        "runServerAs": "modelservice",
        "reuseServerProcesses": "true",
        "serverMinAvailable": "1"
    },
    "launchContext": {
        "contextName": "Data Mining launcher context"
    },
    "launchType": "service",
    "authorizedGroups": [
        "Modellers"
    ]
}

Adapt the JSON compute context template to your needs:

- Change /path/to/file/prestarted_reusable_modelservice_cc.json to whatever file path and file name you wish to contain your JSON compute context definition
- Set our preferred name and description for this new compute context
- Provide the name of the service account you created in place of modelservice in the runServerAs attribute
- Optionally, you may wish to add further attributes:
  - One that is quite useful is "serverInactiveTimeout": "900". From the documentation reference in the next bullet: "Is used only when the reuseServerProcesses attribute is set. The value is the time in seconds that an inactive server remains running before being terminated. An inactive server is defined as a server without an active compute session.". When idle, you will see the pre-started compute servers terminate at the end of such an inactive period - and new ones are restarted if this brings the total of compute servers in the context below the number specified by serverMinAvailable.
  - The Server Contexts section in SAS Help Center has a list of compute server attributes
- Set the value of the serverMinAvailable attribute to the initial number of compute servers you wish to have pre-started. See note below.
- Choose the most appropriate launcher context for the value of the contextNameattribute:
  - For SAS Model Studio use, leave it set to "Data Mining launcher context"
  - For SAS Studio use, change it to "SAS Studio launcher context"

- Either
  - change the group name in the authorizedGroups section from Modellers to the groupID of the group you want to have access to this compute context, or
  - if you do not want to restrict access to this compute context, remove the authorizedGroups section - and the comma at the end of "launchType": "service" - to make this compute context available to all authenticated users.

Note: Remember that pre-started compute servers will consume some resources on your Kubernetes cluster's compute nodes, and as explained above will also reserve resources for potential use, so that they cannot be used for running other pods. The optimal serverMinAvailable number for the workload your users are running is something you may need to find by trial-and-error. I have seen a use case of a small environment with a couple of users who ran relatively small model pipelines, where 4 was a good value for serverMinAvailable. I have also seen another use case of a large shared classroom server used for teaching classes of students how to design models in SAS Model Studio, where 25 was was the chosen value for serverMinAvailable. This was the use case Rob used for the example values in his post. It is a value that you should tune, over time and with experience of what works best for your organization's workload.

Save your changes to /path/to/file/prestarted_reusable_modelservice_cc.json.

Step 3: Create the new compute context with pre-started compute servers

To create the new compute context, use the sas-viya command-line interface, still authenticated as an administrator, to run this command:

sas-viya compute contexts create -r -d @/path/to/file/prestarted_reusable_modelservice_cc.json

Note: the @ symbol in front of the file path is correct and required. You can of course create - and update, and delete - a new compute context interactively in SAS Environment Manager. My aim in this post is to show how to do it from the command line.

Step 4: See if compute servers have been pre-started

If you have access to a visual interface which shows pods running in your SAS Viya namespace in your cluster, you should see the number of sas-compute pods increase. However, depending on the interface, it may be difficult to distinguish the sas-compute pods started under this compute context (and running as the modelservice account) from other compute pods.

If you have access to the kubectl command-line interface for Kubernetes and a kube config file that lets you access your SAS Viya deployment's cluster, this command can show sas-compute pods launched specifically by the SAS Compute service (as opposed to by a user in SAS Model Studio or SAS Studio), and running as the modelservice account:

kubectl get pod -l launcher.sas.com/requested-by-client=sas.compute,launcher.sas.com/username=modelservice

Tip: Scroll the textbox above to the right if necessary, to see how the username who requested the pod is specified as a filter to kubectl get pods.

Here are some example results, when my new compute context's serverMinAvailable value was set to "1", and I had no other SAS Compute sessions running in this compute context or as modelservice:

NAME                                                            READY   STATUS    RESTARTS   AGE
sas-compute-server-b91b1b9f-8827-4b57-9535-160cbf6e2cc4-t8mgl   1/1     Running   0          25s

Possible reasons you might not see the new compute servers starting up include:

You need to wait a little longer - a minute or two is usually enough
The credentials for the service account are incorrect - try deleting and re-creating it
There is not currently enough available CPU or memory to start new pods on Kubernetes nodes in your compute node pool - you may see fewer compute servers starting than you requested
The account you specified in the runServerAs attribute is already running the maximum number of launched SAS Programming Run-time Environment sessions allowed, as set by the SAS_LAUNCHER_USER_PROCESS_LIMIT setting. See my blog post: Limit a user’s simultaneous compute server processes in Viya 2021.1 and later for more on this.

Assuming you did see the new sas-compute server pods starting up, you can now see if they improve performance of your model pipelines.

Use the new compute context in SAS Model Studio

Sign in to SAS Model Studio. If you limited access to the new compute context to only members of a specific group, sign in to SAS Model Studio as a user who is in that group.

Then, create or open a model pipeline. To change the compute context used to run each step of this model pipeline, click the 'Settings' icon, in the top right corner of the project tab.

In the Project Settings dialog, choose Compute Context, and switch from the default 'Data Mining compute context' to your new compute context which pre-starts compute servers:

Select any image to see a larger version.
Mobile users: To view the images, select the "Full" version at the bottom of the page.

I do not believe it is possible to configure individual steps in a model pipeline run under different compute contexts. The whole pipeline runs under the same compute context.

In the screenshot above, you can just about see a four-step sequential pipeline, created from a gradient boosting model in SAS Visual Analytics. When this project runs, each step runs in its own separate compute context. Some models run multiple steps at the same time. You should notice a significant improvement in the total 'wall clock' time (as compared to CPU time) when running these models while sufficient pre-started compute servers in the chosen compute context are available and not currently in use.

Use the new compute context in SAS Studio

SAS Studio users are generally familiar with switching compute contexts - click on the compute context icon top-right in SAS Studio to change to a different context:

If you select a compute context which has unused (i.e. available) pre-started compute servers, you should see a very noticeable improvement in how quickly your compute session in SAS Studio is ready to run SAS code.

The tradeoff for this is that the compute session runs as the shared service account, and not as you (or rather, as the signed-in user). This may have some impact on your filesystem permissions, if you access anything on the filesystem, such as base or other path-based libraries, filerefs etc. But the tradeoff may be worth it for the much faster compute session startup time.

So now you know how to script configuration of the compute context from the command line, and can see how easily it could be automated as part of an environment configuration script. Hopefully, that's useful to someone out there.

Thank you to Rob Collum for reviewing this post and for his great comments. Any remaining errors or omissions are mine.

See you next time!

Find more articles from SAS Global Enablement and Learning here.

thesasuser · ‎11-06-2024

In the step 1 a account has been created (-u modelservice -p mySecretPassword123).
Can you please clarify if this account is the one that already exists with authentication provider
OR

is it an account created for this purpose and need not exist with the authentication provider. Something like internal account in SAS 9.x.

SASKiwi · ‎11-06-2024

As I understand it this is the key issue from the above:

"A compute context you configure to have pre-started available compute servers must run those compute servers under a shared service account"

This service account must be defined in the OS (via LDAP etc) as OS processes are run under it. In any case there is no equivalent of internal accounts in Viya because there is no equivalent of SAS 9 metadata repositories.

thesasuser · ‎11-12-2024

Thanks @SASKiwi for the response.
Wondering if there are any cost implications when we have a large number of pre started compute servers?

SASKiwi · ‎11-12-2024

@thesasuser - I suspect the main cost implication would be increased memory usage as each compute server instance will consume memory regardless of whether it has been used or not. I wouldn't expect CPU usage to change much as the overhead of starting a compute server would be low compared to when it is used for processing.