Add statements to an existing SAS Viya compute context autoexec
SAS Administrators use autoexec code blocks in both the compute service configuration which applies to all compute sessions, and in compute contexts which apply only to compute sessions started under that context, to pre-configure their users' SAS Programming Run Time sessions in various ways. Autoexec code can be used to define commonly-used library references, macros, SAS system options, lockdown paths and more.
My last post here presented a script to add statements such as lockdown paths and librefs to SAS compute service autoexec. This post presents a similar script, which does the same for an existing compute context.
Modifying an existing compute context is very easy to do interactively using SAS Environment Manager. And it's also easy to create entirely new compute contexts with properties set as you wish using the SAS Viya CLI. But sometimes, SAS Administrators would prefer to script and automate this kind of configuration, including changes to existing compute contexts in a SAS Viya deployment.
Updating an existing compute context from the command line has just become easier with a new pyviyatool, updatecomputecontext.py.
Like the script presented in the last post, the script presented in this one tries to be idempotent: it checks whether the lines of SAS code to be added are already in the specified compute context's autoexec code block. If they are not it adds them and updates the compute context with the resulting JSON file (using the new updatecomputecontext.py pyviyatool). The way autoexec code is stored in the JSON representation of a compute context is slightly different from the way it is stored in the JSON representation of the compute service autoexec_code configuration instance, and this script accounts for that slight difference.
Creating an entirely new compute context, complete with autoexec lines if desired, is fairly easy in a script. You can create a JSON file named e.g. /tmp/compute_context.json containing the desired compute context definition in any of several ways (from a template, download the JSON for an existing context etc.). Then you can pass that JSON file into a command like this:
sas-viya compute contexts create -r -d @/tmp/compute_context.json
What we didn't have until now was a neat method to modify an existing compute context from the command line. There was no such command as sas-viya compute contexts update in the compute plugin to the SAS Viya CLI at the time of writing. So we cannot update the compute context with a command like this:
sas-viya compute contexts update -r -d @/tmp/compute_context_config_with_autoexec_updated.json
As of SAS Viya 2024.10, sas-viya compute contexts has sub-commands for create, delete, generate-template, list, show and validate. But there is no update sub-command, which is why I wrote the new pyviyatool.
The closest thing you could do previously, using just the SAS Viya CLI and without writing code that calls REST APIs, was to delete an existing compute context and then create a new one of the same name from a JSON file. That can work, but results in the compute context resource ID changing, which can create its own issues with other automation that relies on the context ID. Modifying an existing context is preferable.
Update a compute context autoexec from a bash script
Define add_code_to_compute_context_autoexec()
Below is a custom bash function called add_code_to_compute_context_autoexec() which adds one or more lines of SAS code to the existing autoexec code block in an existing compute context, if the new code lines are not already part of the existing compute service autoexec. It uses the SAS Viya Command-Line Interface, the JSON query utility jq and the new pyviyatool, updatecomputecontext.py.
add_code_to_compute_context_autoexec ()
{
# Delete any old working files from previous runs: /tmp/compute_*.json, /tmp/compute_*.txt
rm -f /tmp/compute_*.json /tmp/compute_*.txt
compute_context_name=${1}
file_code_to_add=${2}
# Test that the first input parameter is the name of a SAS compute context, and returns valid JSON document
sas-viya --output fulljson compute contexts show -n "${compute_context_name}" 2>&1 > /tmp/compute_context_config.json
# cat /tmp/compute_context_config.json
if [ -s /tmp/compute_context_config.json ]; then
# Something was written to the file - is it JSON?
retval=$(jq -re '""' /tmp/compute_context_config.json 2>&1)
# echo "Found compute context named ${compute_context_name} in SAS Viya"
if [ ! -z "${retval}" ]; then
echo "Error: invalid JSON while getting \"${compute_context_name}\" using sas-viya CLI - ${retval}."
return 1
fi
else
echo "Error: Compute context \"${compute_context_name}\" not found."
return 1
fi
# Test that the second input parameter is populated and the file path contains exists
if [ -f ${file_code_to_add} ]; then
# echo "File \"${file_code_to_add}\" exists"
# Delete some of the parts of compute_context_config.json we won't need.
# (It may still contains some parts we don't need, but this is clean enough to be readable.)
jq -r 'del( .links )' /tmp/compute_context_config.json > /tmp/compute_context_config_clean.json
# cat /tmp/compute_context_config_clean.json
# Many compute contexts, and all out-of-the-box compute contexts do not have any autoexec code
# Therefore, they do not have the keys for one in their JSON definition.
# Check whether the key for an autoexec code block exists. If it does, get the autoexec code from it.
# If it does not, create it.
if [[ $(jq -e '.environment | has("autoExecLines")' /tmp/compute_context_config_clean.json) == "true" ]]; then
echo 'Compute context has autoExecLines'
jq -r '.environment.autoExecLines[]' /tmp/compute_context_config_clean.json > /tmp/compute_context_autoexec.txt
cp /tmp/compute_context_config_clean.json /tmp/compute_context_config_with_autoexec.json
else
echo 'Compute context does not have autoExecLines, adding an empty array for them'
jq -r '.environment |= {"autoExecLines": []} + . ' /tmp/compute_context_config_clean.json > /tmp/compute_context_config_with_autoexec.json
# cat /tmp/compute_context_config_with_autoexec.json
rm -f /tmp/compute_context_autoexec.txt
touch /tmp/compute_context_autoexec.txt
fi
# cat /tmp/compute_context_config_with_autoexec.json
# cat /tmp/compute_context_autoexec.txt
# Convert the multiline content of ${file_code_to_add} to a single line
# with a \n for each newline, for use in the test below to see if it is already in the existing autoexec.
cat ${file_code_to_add} | sed -n 'H;${x;s/^\n//;s/\n/\\n/g;p}' > /tmp/compute_autoexec_code_to_add_oneline.txt
# cat /tmp/compute_autoexec_code_to_add_oneline.txt
# Convert the multiline content of /tmp/compute_context_autoexec.txt to a single line
# with a \n for each newline, for use in the test below.
cat /tmp/compute_context_autoexec.txt | sed -n 'H;${x;s/^\n//;s/\n/\\n/g;p}' > /tmp/compute_context_autoexec_oneline.txt
# cat /tmp/compute_context_autoexec_oneline.txt
# Load strings containing the context's existing autoexec code and the candidate code to add to it in variables.
compute_context_autoexec_code_oneline=$(cat /tmp/compute_context_autoexec_oneline.txt)
compute_context_autoexec_code_to_add_oneline=$(cat /tmp/compute_autoexec_code_to_add_oneline.txt)
# echo ${compute_context_autoexec_code_oneline}
# echo ${compute_context_autoexec_code_to_add_oneline}
# If compute_autoexec_code_oneline.txt does not already contain compute_autoexec_code_to_add_oneline.txt, add it. If it already contains that block of code verbatim, do nothing.
if [[ ${compute_context_autoexec_code_oneline} == *"${compute_context_autoexec_code_to_add_oneline}"* ]]; then
echo "The context's autoexec code block already contains that code. Nothing to do."
else
echo "The context's autoexec code block does not yet contain that code. Adding it."
# Append the new autoexec code to the existing autoexec code block
rm -f /tmp/compute_context_autoexec_code_multiline_final.txt
cat /tmp/compute_context_autoexec.txt \
${file_code_to_add} > /tmp/compute_context_autoexec_code_multiline_final.txt
# cat /tmp/compute_context_autoexec_code_multiline_final.txt
# Convert it back to a JSON array
jq -R -s 'split("\n")' < /tmp/compute_context_autoexec_code_multiline_final.txt > /tmp/compute_context_autoexec_array.txt
# cat /tmp/compute_context_autoexec_array.txt
# Replace the existing autoexec lines array in the JSON for the compute context with this new one
autoexec_array=$(cat /tmp/compute_context_autoexec_array.txt)
# echo $autoexec_array
jq -r --argjson autoexec_array "$autoexec_array" '.environment.autoExecLines = $autoexec_array' /tmp/compute_context_config_with_autoexec.json > /tmp/compute_context_config_with_autoexec_updated.json
# cat /tmp/compute_context_config_with_autoexec_updated.json
# There is no such command as "sas-viya compute contexts update" in the compute plugin to the SAS Viya CLI at the time of writing.
# So we cannot update the compute context with a command like this:
#
# sas-viya compute contexts update -r -d @/tmp/compute_context_config_with_autoexec_updated.json
#
# "sas-viya compute contexts" has sub-commands for create, delete, generate-template, list, show and validate.
# But at the time I wrote this example, there is no "update" sub-command, so I wrote a pyviyatool to update compute contexts.
# Update the compute context using the updatecomputecontext.py pyviyatool
${pyviyatools_path}/updatecomputecontext.py -n "${compute_context_name}" -f /tmp/compute_context_config_with_autoexec_updated.json
fi
else
echo "File \"${file_code_to_add}\" not found"
fi
}
Some notes:
- add_code_to_compute_context_autoexec()relies on the SAS Viya Command-line Interface (sas-viya), and expects to find it from the command sas-viya. So, before you use the function in this post, you must install and configure the CLI to connect to your SAS Viya deployment, and then authenticate as a user who is a SAS Administrator. This has been written about a lot, not least in these posts:
- Gerry Nelson: SAS Viya Command-Line Interface for Administration (update)
- Gerry Nelson: Overview of installing and using the SAS Viya containerized CLI
- Stuart Rogers: SAS Viya CLI with Group-Managed Service Accounts
- If the sas-viya executable or an alias or link to it is not in your path, you may need to modify the function to add the path to sas-viya in your environment, or provide it in an environment variable.
- The function uses jq: https://github.com/jqlang/jq.
- The function uses a new pyviyatool,updatecomputecontext.py. If you have not installed the pyviyatools, see the project README.md or INSTALL.md instructions in GitHub. If you already have the pyviyatools installed from before 18 November 2024, you may need to run a git pull or similar command to get updatecomputecontext.py.
- The function deletes and creates a set of text files (containing JSON and plain text) in the /tmp directory, named like /tmp/compute_*.json and /tmp/compute_*.txt. If this might cause an issue in your environment, feel free to modify the function as you see fit to store the temporary files somewhere else.
- There are quite a few commented-out cat and echo statements, which were useful to me during the function's development. I left them in the script in case they prove useful to you.
The function expects two (positional) parameters. The first is the name of the compute context to be updated. The second is the path to a file containing one or more lines of new SAS code, to be added to that compute context's autoexec code lines.
Call add_code_to_compute_context_autoexec()
To call the function (after authenticating against the SAS Viya CLI and doing the other things in the bullet points just following the script above), simply create a text file containing the SAS code statements you wish to add (if they are not already in the chosen compute context's autoexec code block), and call the function passing context name, and the path to that text file as parameters:
# Name of compute context to update
compute_context_name="SAS Studio compute context"
# Then, create a file containing the SAS code you want to add to an autoexec if it doesn't
# already contain this code.
tee /tmp/autoexec_code_to_add.txt > /dev/null << EOF
lockdown path='/shared_path';
libname shrdata '/shared_path/data';
EOF
# cat /tmp/autoexec_code_to_add.txt
add_code_to_compute_context_autoexec "${compute_context_name}" "/tmp/autoexec_code_to_add.txt"
Depending on whether the compute context already has SAS code lines in its autoexec or not, the first line of expected output may either be:
Compute context has autoExecLines
or:
Compute context does not have autoExecLines, adding an empty array for them
Then the rest of the expected output will look something like this:
The context's autoexec code block does not yet contain that code. Adding it.
Compute context: 'SAS Studio compute context' found in SAS Viya deployment with id: 9f0c2e7e-0cd3-4d01-bcd8-8230611119d2
Found 'SAS Studio compute context' in input file '/tmp/compute_context_config_with_autoexec_updated.json'
The compute context id '9f0c2e7e-0cd3-4d01-bcd8-8230611119d2' in the input file matches the one in the SAS Viya deployment.
Context updated.
If you were to run the function a second time, with the same content in the file you pass as a parameter, you should instead see this:
Compute context has autoExecLines
The context's autoexec code block already contains that code. Nothing to do.
The example autoexec code statements shown above are:
lockdown path='/shared_path";
libname shrdata '/shared_path/data';
These statements would make sense for a SAS Viya deployment in which you have mounted a volume into your compute server (and other SAS Programming Runtime) pods, which makes the filesystem path /shared_path available to processes running in those pods. I discussed this in my previous post and won't duplicate that discussion here.
You could add any valid and relevant SAS code statements to your equivalent of my example /tmp/autoexec_code_to_add.txt. Anything that makes sense in an autoexec code block should work.
As with the script in my previous post, be consistent with your indentation, trailing spaces and whitespace in general, if you care about the idempotency part of the function: that it won't add code that already exists in the compute context autoexec. The function doesn't do anything particularly clever about trimming white space, so for example if your compute service autoexec contains a line indented by one space, and you run the function to add the same line not indented, you should expect the 'duplicate' line to be added to the autoexec, and possibly some warnings in the server logs when compute sessions start up.
Update a compute context autoexec interactively, in SAS Environment Manager
If you only need to add code to a compute context autoexec once, and you don't need to automate this task, do it interactively in SAS Environment Manager. This is documented in SAS Help Center, in the SAS Environment Manager User's Guide, in the Contexts Page section under Create a Compute Context and Edit a Context.
Sign in as an administrator and navigate to the Contexts page. (Users who are not members of the SAS Administrators custom group don't see the Contexts page). Choose Compute Contexts from the View dropdown, then select a compute context, e.g. SAS Studio Compute Context. Click the Edit button on the right-hand side of the application page, to edit the context.
SAS Environment Manager's Contexts page, showing compute contexts and the edit button.
Select any image to see a larger version.
Mobile users: To view the images, select the "Full" version at the bottom of the page.
In the Edit Compute Context dialog, switch to the Advanced tab. The field labelled "Enter each autoexec statement on a new line:" contains the autoexec code statements that are run as compute sessions under this compute context start:
SAS Environment Manager, Context page, editing a compute context. The Advanced tab of the Edit Compute Context dialog has a field
containing lines of autoexec code for this context.
Adding your desired lines of SAS code to be automatically executed when compute sessions under this context start is all you need to do if you don't need to script anything. But for those of us who look after automatically-deployed and automatically-updated environments, the add_code_to_compute_context_autoexec() bash function does something we could not have done easily before.
See you next time!
Find more articles from SAS Global Enablement and Learning here.
