Did you know there is a new command line interface in the new SAS Viya (2020.1 and later)?
To run in batch a .sas program called 1.sas
:
In the new SAS Viya (2020.1 and later) you could use the Command-Line Interface (CLI) batch plug-in:
/opt/sas/viya/home/bin/sas-viya --profile ${SAS_CLI_PROFILE} batch jobs submit-pgm --pgm-path /tmp/1.sas --context default --watch-output --wait-log-list --results-dir /tmp
vs. SAS Viya 3.5 where you had to write:
/opt/sas/spre/home/SASFoundation/sas -autoexec "/opt/sas/viya/config/etc/workspaceserver/default/autoexec_deployment.sas" /tmp/1.sas -log /tmp/1.log
SAS Batch Command Line Utility is a command line interface that enables you to submit SAS programs or commands from a command line to a SAS Viya environment running in a Kubernetes cluster for batch processing. The principles are described Command-Line Interface: Overview.
The top-level command is sas-viya
. It is used to initialize, authenticate, and execute the batch
plug-in. For a complete plug-in list, see Command-Line Interface: Plug-Ins.
There are some extra steps before you can submit your sas-viya batch jobs submit-pgm
command.
Find your Kubernetes namespace. In this example, SAS Viya is running in the gelcorp
namespace. If you don't know in which namespace Viya is running, type:
kubectl get ns
The SAS Viya applications (e.g. SAS Environment Manager, SAS Drive, etc.) can be accessed, by default, at the location described by $INGRESS_URL
.
current_namespace=gelcorp
INGRESS_SUFFIX=$(hostname -f)
INGRESS_URL=http://${current_namespace}.${INGRESS_SUFFIX}
echo ${INGRESS_URL}
CAS supports encrypted connections between the server and the clients. You can enable TLS to secure communications between the server and clients. The certificate used for client server communication needs to be signed by a certificate authority (CA) that is trusted by all potential clients.
Therefore, if you have a SAS Viya deployment, where TLS is enabled, you need to execute these commands to get the certificates once. You need a kubectl admin role.
mkdir -p ~/.certs kubectl cp $(kubectl get pod -l app=sas-logon-app -o=jsonpath='{.items[0].metadata.name}'):security/trustedcerts.pem ~/.certs/${current_namespace}_trustedcerts.pem
You need to run the following commands every time, before the sas-viya
profile
and auth
commands. These will point the CLI to the location of the certificates and certificate authority bundle (TLS).
export SSL_CERT_FILE=~/.certs/${current_namespace}_trustedcerts.pem
export REQUESTS_CA_BUNDLE=${SSL_CERT_FILE}
Switch to the folder where the SAS VIYA CLI is installed. I assume you installed it in the $clidir
.
clidir=/opt/sas/viya/home/bin
cd $clidir
Create a functional profile called ${SAS_CLI_PROFILE}
. This resolves to gelcorp
.
echo ${current_namespace} export SAS_CLI_PROFILE=${current_namespace} ./sas-viya --profile ${SAS_CLI_PROFILE} profile set-endpoint "${INGRESS_URL}" ./sas-viya --profile ${SAS_CLI_PROFILE} profile toggle-color off ./sas-viya --profile ${SAS_CLI_PROFILE} profile set-output fulljson # login and create a token ./sas-viya --profile ${SAS_CLI_PROFILE} auth login -user sasadm -password *******
By default, your authentication remains active for 12 hours. You can use the sas-viya batch auth logout
command to sign out.
Create a 1.sas in the /tmp folder, for example:
echo 'proc setinit; run; quit;' | tee /tmp/1.sas
Submit the program for execution:
cd /tmp/ /opt/sas/viya/home/bin/sas-admin --profile ${SAS_CLI_PROFILE} batch jobs submit-pgm --pgm-path /tmp/1.sas --context default --watch-output --wait-log-list --results-dir /tmp
.
In SAS Environment Manager, find out your CAS controller host and port.
Select any image to see a larger version.
Mobile users: To view the images, select the "Full" version at the bottom of the page.
Create a 2.sas program and submit it for execution:
echo 'cas casauto host="controller.sas-cas-server-default.gelcorp.svc.cluster.local" port=5570;' | tee /tmp/2.sas
echo 'cas casauto terminate;' | tee -a /tmp/2.sas
cat /tmp/2.sas
Submit it for execution:
cd /tmp/ /opt/sas/viya/home/bin/sas-viya --profile ${SAS_CLI_PROFILE} batch jobs submit-pgm --pgm-path /tmp/2.sas --context default --watch-output --wait-log-list --results-dir /tmp
The log would look like this:
>>> File set created, ID="JOB_20201009_045619_929_1" >>> Uploading "2.sas" >>> Job submitted, ID="64139ca3-84aa-4d6c-bed8-d7c25a3647fd". >>> Waiting for job to complete... >>> Job started. …. 1 cas casauto host="controller.sas-cas-server-default.gelcorp.svc.cluster.local" port=5570; NOTE: The session CASAUTO connected successfully to Cloud Analytic Services controller.sas-cas-server-default.gelcorp.svc.cluster.local using port 5570. The UUID is fc406c5b-3ec4-914c-97b2-48c99121eb1c. The user is sasadm and the active caslib is CASUSER(sasadm). NOTE: The SAS System used: real time 0.40 seconds cpu time 0.06 seconds … <<< Job ended. <<< Downloading files in fileset to "/tmp/JOB_20201009_045619_929_1". <<< Downloading "2.log" <<< Downloading "2.sas" <<< Copying log and list files from "/tmp/JOB_20201009_045619_929_1" to current directory <<< Copying "2.log" <<< Deleting results directory. <<< Deleting batch job. <<< Deleting fileset.
The sas-viya batch CLI seems to function more like the RSUBMIT statement from SAS/Connect. Only the tasks executed are not in a "server" session, but in a Kubernetes cluster, while the results and the output are delivered to the "client" session.
More details on how to Submit SAS Programs in Batch.
To perform the batch commands, you must have the SAS Viya CLI installed on the machine where you will execute the commands. Gerry Nelson provided more context in his post Keeping the SAS Administration Command-Line interfaces up-to-date. His post talks about the sas-admin
executable, which has been replaced by sas-viya
for SAS Viya release 2020.1 and later. Replace sas-admin with sas-viya in his instructions and you should be fine.
To authenticate with an .authinfo file you need the pyviyatools. The pyviatools installation instructions are found here: INSTALL.md .
A more secure way is to avoid encoding credentials. You can use an .authinfo file in conjunction with the pyviya tools. See Gerry's post: Introducing the GEL pyviyatools.
# Instead of
# ./sas-viya --profile ${SAS_CLI_PROFILE} auth login -user sasadm -password *****
# Create .authinfo - replace ***** with real credentials
tee ~/.authinfo > /dev/null << EOF
default user sasadm password *****
EOF
chmod 600 ~/.authinfo
# Browse to PyViya Tools folder and use the loginviauthinfo command for authentification
cd ~/admin/pyviyatools
./loginviauthinfo.py -f ~/.authinfo
The pyviatools installation instructions are found here: INSTALL.md .
To execute sas programs in batch in the new SAS Viya (2020.1 and later), consider using the sas-viya batch CLI. It is not the only option, there are other CLIs that allow you to manage, transfer, automate. What if you have parameters to pass to a SAS program running in batch? Read How to Run SAS Programs with Parameters in Batch in SAS Viya. What if you need to run a SAS Studio flow instead of a SAS program? Read How to Run a SAS Studio Flow in Batch.
Full credits and many thanks to: Ajmal Farzam, Gerry Nelson, Mark Thomas and Stephen Foerster.
Thank you for your time reading this article. If you think the article was useful, like it.
Please comment and share your experience with SAS Viya and help others.
Find more articles from SAS Global Enablement and Learning here.
@Bogdan_Teleuca - This is very informative. Is there a GUI that automates the command line interface in the same way Schedule Manager in SAS Management Console automates the SAS 9 batch job command line interface? SAS Workflow Scheduler perhaps?
I don't think there is one.
@Bogdan_Teleuca - Are there any plans to provide one?
Interesting article. Only two weeks ago I had to come up with some solutions to batch submit jobs in Viya 3.5 from Control-M as part of the DI ETL processing and subsequent load into CAS.
I came up with two solutions.
- The first being a shell script to run a SPRE session in batch as described in the article above.
- The second more interesting one was to use use REST calls. I came up with a working proof of concept where by from a SAS9.4 DI Server using proc http, I made two calls:1) obtain an auth token : https://xxxx.xxxx.x.nz/SASLogon/oauth/token, then followed this with a call to the SAS Viya job execution service. I believe that Control-M can make REST calls (not tested), but otherwise Control-M could be used to call a SAS9.4 job and SAS9.4 can make the REST calls.
Part of the code for the second REST call to kick off the job:
proc http url="&viyahostname/jobExecution/jobs"
method="POST"
in=json_in
out=out;
Headers "Content-Type"= "application/vnd.sas.job.execution.job.request+json"
'Accept'='application/vnd.sas.job.execution.job+json'
'Authorization'="Bearer &access_token"
;
run;
Job information and code is passed in the json_in file and included code, as well as parameters that you could pass. These parameters would appear as macro variables in the SAS session for example (json fragment):
"testparm": "test456",
"testparmB": "test789"
in the SAS Session - shows the parameters appearing as macro variables:
18 +%put _all_;
GLOBAL DTSTAMP 2021_03_18T22_53_23
GLOBAL SYS_COMPUTE_DATA /opt/sas/viya/config/data/compsrv/default
GLOBAL SYS_COMPUTE_JOB_ID 89EF29E8-ECCD-6C48-802D-D2146A658913
GLOBAL SYS_COMPUTE_SESSION_ID 259db425-7750-4c90-8d25-f47e5cb8fcf8-ses0000
GLOBAL SYS_COMPUTE_SESSION_OWNER mark
GLOBAL SYS_JES_JOB_URI %nrstr(/jobExecution/jobs/be57e9e9-e7b1-4370-8e54-14297f3997af)
GLOBAL TESTPARM test456
GLOBAL TESTPARMB test789
GLOBAL _CASPORT_ 5570
GLOBAL _CONTEXTNAME %nrstr
I kept the SAS code in the job definition simple - being a couple of statements to do a %include. The included .sas file could then contain more extensive code - in this case CAS table load, save, promote etc. code.
The job would display in the Environment Manager - Jobs screen. Like this one (from our Viya 3.4 AWS playpen):
This REST call solution meant that the batch jobs could be called from SAS9 and therefore be made into a SAS9 DI Job and be scheduled using the usual SAS9 scheduling tools.
In this particular application, the existing SAS DI environment was an older - pre M5 release which meant that integration with Viya was limited. We couldn't use proc casutil or proc cas, for example to load data into CAS as part of the nightly ETL processing. Using proc http and REST calls meant that we could kick off a job on Viya that would load data into CAS after the nightly ETL batch had completed.
To develop this, I used simple CURL statements and Postman to test the REST calls
Mark
Thank you BogdanT and MarkBodt_NewZea
Sorry for a naive question; I don't have access to a Viya environment. The above seems like a good bit of work just to batch submit. For a casual Viya batch submit, is the SAS Studio background submit a reasonable option? Obviously it's not CLI, but for most of my use cases, I'd be happy to manually open a program and click a "batch submit" button, as long as I got a logical batch submit (i.e. starts a new clean session, executes the program, and writes a .log file and .lst file).
I see on a Viya 4 presentation roadmap that job scheduling equivalent to Platform Services / LSF is due in Viya 2021.2. Is this likely to fix the missing graphical user interface issue?
Are you ready for the spotlight? We're accepting content ideas for SAS Innovate 2025 to be held May 6-9 in Orlando, FL. The call is open until September 16. Read more here about why you should contribute and what is in it for you!
Data Literacy is for all, even absolute beginners. Jump on board with this free e-learning and boost your career prospects.