The Easiest Way to Run a SAS Studio Flow in Batch

2 Likes

Have you wondered how to run, in batch mode, a SAS Studio flow? This post provides you the latest approach and a few useful scripts. Since the Long-Term Support (LTS) release 2023.03 (May 2023), you can use the deploy - redeploy options available in SAS Studio. These options allow users to have the control over the code deployed. We can say SAS Studio flows execution has been made DevOps ready.

The present blog overrides an older post How to Run a SAS Studio Flow in Batch. If your sole purpose is to execute a SAS Studio flow in batch, it is no longer necessary to generate the code from a flow, as described in How To Get the Code From a SAS Studio Flow and Why It Matters or How to Get the Code From a SAS Studio Flow Using Python. Unless you have another use case that justifies it, for example versioning the code, altering it, "transporting" it, etc.

Deploy as a Job - Execute - Change - Redeploy

It's easy, watch the following video and you will see why.

(view in My Videos)

Execute the Job

To execute a job, in batch, you can use the SAS Viya CLI.

Authenticate to SAS Viya and Get an Access Token

# Set Variables

# SAS Viya namespace - optional, but used as a prefix elsewhere
export current_namespace=gelenv
export SAS_CLI_PROFILE=${current_namespace}
# SAS Viya PEM file for TLS auth
export SSL_CERT_FILE=~/.certs/${current_namespace}_trustedcerts.pem
# user and pass
USER=myuser@mytenant.com
PASS=mypass
# SAS Viya URL
export INGRESS_URL=https://fill_in_here
# SAS Viya CLI install location
clidir=/opt/sas/viya/home/bin

# Create a SAS Viya CLI Profile
$clidir/sas-viya --profile ${SAS_CLI_PROFILE} profile set-endpoint ${INGRESS_URL}
$clidir/sas-viya --profile ${SAS_CLI_PROFILE} profile toggle-color on
$clidir/sas-viya --profile ${SAS_CLI_PROFILE} profile set-output fulljson
# Show profile
$clidir/sas-viya --profile ${SAS_CLI_PROFILE} profile show
# Login in
$clidir/sas-viya --profile ${SAS_CLI_PROFILE} auth login -user $USER -password $PASS
unset PASS
# Login succeeded. Token saved.

There are more elegant and secure approaches to managing credentials. Read further, to find out.

List the Existing Jobs

Filter by the name of the job, then retrieve the id in a variable jid.

# install jq if needed
sudo apt install jq -y

jid=$($clidir/sas-viya --profile ${SAS_CLI_PROFILE} --output json job requests list --filter 'eq(name,'Flow')' | jq -r '.items[]["id"]')
echo "The SAS object is wrapped in this job request: $jid"

Execute the Job Request

With job requests execute, you are executing the latest version of the deployed code from your SAS Studio flow.

$clidir/sas-viya --profile ${SAS_CLI_PROFILE} job requests execute --id $jid

You must have the SAS Viya CLI installed on a machine where you will execute the commands. The CLI version 1.21.17 has been used in the examples that follow. For a complete plug-in list, see Command-Line Interface: Plug-Ins. The installation instructions can be found here: SAS® Viya®: Using the Command-Line Interface.

Managing Change

What happens if you change something in a SAS Studio flow?

Save it.
Redeploy it.

"That's all, folks!"

Select any image to see a larger version.
Mobile users: To view the images, select the "Full" version at the bottom of the page.

Execution Results and Logs

After posting, I received a few queries: "What about the logs, the status, the execution results? Is there a way to obtain them?"

I decided to do some extra investigations. The conclusions:

The SAS® Viya® Platform Administration documentation specifies that as of 2023.09 (September 2023) stable version:

The --wait-results option is available as of release 2.6.44 of the job CLI plug-in. You can use this option with the job requests execute and job flows execute commands.

The option enables the job CLI plug-in to wait for the results from the program associated with the job request or job flow. The CLI polls the program until the CLI receives the program results. The results are written to stdout. For an example of using this option, see Capture Results from a Program or Command in SAS Viya Platform: Jobs and Flows.

I could not find an option to get the full log, just like we do when we execute using the batch submit-pgm command.

Think Security

Avoid hardcoding credentials in your code files. You can use an:

An .authinfo file in conjunction with the pyviya tools. See Gerry's posts: Introducing the GEL pyviyatools.
If you are using SAS Viya on Azure, use Azure Key Vault to store your secrets.

Conclusions

To execute in batch SAS Studio flows, from SAS Viya 2023.03 or later, consider using the sas-viya job requests CLI and the deploy, redeploy flow SAS Studio options.

Resources

For more information, see the following information:

Thank you for your time reading this post. If you liked the post, give it a thumbs up. Please comment and tell us what you think about running flows in batch. If you wish to get more information, please write me an email.