topic Re: Running several SAS Program Files at once in SAS Programming

Running several SAS Program Files at once

Jack1 — Tue, 22 Nov 2016 18:21:39 GMT

I have several SAS programs in a folder that I run in sequence. However I would like to create a batch program to run these SAS program files one-by-one instead of manually.

How would I code this?

Thanks

Jack

Re: Running several SAS Program Files at once

PeterClemmensen — Tue, 22 Nov 2016 18:42:15 GMT

Use Include statements 🙂

http://support.sas.com/documentation/cdl/en/lrdict/64316/HTML/default/viewer.htm#a000214504.htm

Re: Running several SAS Program Files at once

TheShark — Tue, 22 Nov 2016 19:38:44 GMT

It sounds like you want to fully automate the process but keep the code separated, possibly for individual logs/troubleshooting/future changes? Regardless, I think what I would do is create a batch file and add it to the Windows task scheduler.

The batch file (.bat) would look something like this and would generate a date stamped log for each run,

Start/w "Auto Metrics" "C:\Program Files\SASHome\SASFoundation\9.4\sas.exe" -CONFIG "C:\Program Files\SASHome\SASFoundation\9.4\nls\en\sasv9.cfg" -sysin "Z:\blahblahblah\auto_metrics_code.sas" -log "Z:\blahblahblah\auto_metrics_log_%date:~-4,4%%date:~-7,2%%date:~-10,2%.txt" -print "Z:\blahblahblah\auto_metrics_log_%date:~-4,4%%date:~-7,2%%date:~-10,2%.lst"

Start/w "Auto Metrics" "C:\Program Files\SASHome\SASFoundation\9.4\sas.exe" -CONFIG "C:\Program Files\SASHome\SASFoundation\9.4\nls\en\sasv9.cfg" -sysin "Z:\blahblahblah\auto_metrics_code_STEP2.sas" -log "Z:\blahblahblah\auto_metrics_log_STEP2_%date:~-4,4%%date:~-7,2%%date:~-10,2%.txt" -print "Z:\blahblahblah\auto_metrics_log_STEP2_%date:~-4,4%%date:~-7,2%%date:~-10,2%.lst"

Start/w "Auto Metrics" "C:\Program Files\SASHome\SASFoundation\9.4\sas.exe" -CONFIG "C:\Program Files\SASHome\SASFoundation\9.4\nls\en\sasv9.cfg" -sysin "Z:\blahblahblah\auto_metrics_code_STEP3.sas" -log "Z:\blahblahblah\auto_metrics_log_STEP3_%date:~-4,4%%date:~-7,2%%date:~-10,2%.txt" -print "Z:\blahblahblah\auto_metrics_log_STEP3_%date:~-4,4%%date:~-7,2%%date:~-10,2%.lst"

Once you have the batch file working, just schedule it in the task scheduler at whatever cadence you see fit.

http://support.sas.com/documentation/cdl/en/hostwin/67962/HTML/default/viewer.htm#p16esisc4nrd5sn1ps5l6u8f79k6.htm

Re: Running several SAS Program Files at once

SASKiwi — Tue, 22 Nov 2016 20:01:36 GMT

If you are running your programs on a SAS server you may want to consider using the batch and scheduling SAS jobs capabilities of SAS Management Console. It takes care of constructing correct SAS batch command lines without you having to construct them manually.

Re: Running several SAS Program Files at once

Kurt_Bremser — Tue, 22 Nov 2016 20:16:28 GMT

On UNIX, I'd do

sas sas1.sas&
sas sas2.sas&
sas sas3.sas&
wait

The wait command will only finish once all the background jobs have finished.

Re: Running several SAS Program Files at once

Jack1 — Wed, 23 Nov 2016 15:53:08 GMT

Great thanks.

Re: Running several SAS Program Files at once

KenMac — Tue, 21 Aug 2018 19:15:50 GMT

Hi,

I have another use case for this question. I have a set of SAS programs that do share any dependencies that I wish to run in parallel. At the end of the set, I wish to trap the return code from each job and fail the parent process if any of the child processes report an error.

I came up with the attached script, but it does not work in the way I need. Per the man page for wait, once the child process terminates, the wait statement refers to a terminated process and Unix (Solaris in my case) returns a zero. If I order the scripts in order the the amount of execution time required, then it works but I cannot predict precisely the completion order for all the steps in the job.

Any suggestions on how to get this to work without pre-knowledge of the order of completion?

In this example, the script runs 5 SAS programs. Each one just sleeps for a set number of seconds and then aborts with a return code equal to the number of seconds the program sleeps. The number of seconds is in the name of the program.

Thanks in advance for your help!

#!/bin/sh

# Test the wait command

echo "Test RC Start "  `date`
 
SASEXE=/apps/sasconfig/Lev1/SASApp/sas.sh
UTC=\"`date -u +%Y-%m-%dT%H:%M:%S.000+0000`\"
TESTsrc=/data/testrc

# Launch TestRC jobs

$SASEXE -sysin $TESTSRC/test_rc_50.sas -set utc $UTC -sysparm none -log $TESTSRC/test_rc_50.log -noterminal -logparm WRITE=IMMEDIATE &
test_50_pid=$!

$SASEXE -sysin $TESTSRC/test_rc_40.sas -set utc $UTC -sysparm none -log $TESTSRC/test_rc_40.log -noterminal -logparm WRITE=IMMEDIATE &
test_40_pid=$!

$SASEXE -sysin $TESTSRC/test_rc_30.sas -set utc $UTC -sysparm none -log $TESTSRC/test_rc_30.log -noterminal -logparm WRITE=IMMEDIATE &
test_30_pid=$!

$SASEXE -sysin $TESTSRC/test_rc_20.sas -set utc $UTC -sysparm none -log $TESTSRC/test_rc_20.log -noterminal -logparm WRITE=IMMEDIATE &
test_20_pid=$!

$SASEXE -sysin $TESTSRC/test_rc_10.sas -set utc $UTC -sysparm none -log $TESTSRC/test_rc_10.log -noterminal -logparm WRITE=IMMEDIATE &
test_10_pid=$!

# Wait for all programs to complete

wait $test_50
test_50_rc=$?

wait $test_40
test_40_rc=$?

wait $test_30
test_30_rc=$?

wait $test_20
test_20_rc=$?

wait $test_10
test_10_rc=$?

#  Staging Phase 1 programs are complete - display all return codes

echo "Test RC Complete "  `date`
echo " "

echo "Test PID and RC Return Codes"
echo "__________________________________________"
echo "  test_50   $test_50_pid   $test_50_rc    "
echo "  test_40   $test_40_pid   $test_40_rc    "
echo "  test_30   $test_30_pid   $test_30_rc    "
echo "  test_20   $test_20_pid   $test_20_rc    "
echo "  test_10   $test_10_pid   $test_10_rc    "

#  Set a global return code 

if (( test_50_rc          <= 1 &&  \
      test_40_rc          <= 1 &&  \
      test_30_rc          <= 1 &&  \
      test_20_rc          <= 1 &&  \
      test_10_rc          <= 1     )); then
  echo "All programs completed sucessfuly"
  rval=0
else
  echo "There was an problem in the test process.  Kindly review the logs for details"
  rval=2
fi

echo "Test RC Ending "  `date`
echo "Return Code = $rval"
exit $rval

Re: Running several SAS Program Files at once

SASKiwi — Tue, 21 Aug 2018 22:25:46 GMT

Parallel processing is much more easily achieved if you have SAS/CONNECT available to you. If you do then you can do it all in your SAS code and don't need a controlling script at all.

Re: Running several SAS Program Files at once

Kurt_Bremser — Wed, 22 Aug 2018 07:01:53 GMT

This is a test I just set up on AIX with ksh and bash:

Script testback1:

sleep 1
exit 1

Script testback2:

sleep 2
exit 2

Script testback:

./testback1&
tb1=$!
./testback2&
tb2=$!
wait $tb2
rc2=$?
wait $tb1
rc1=$?

echo RC1=$rc1
echo RC2=$rc2

You can see that I check in reverse sequence, so the "shorter" script has already finished when I test the "longer" one.

The result of running testback is this:

$ ./testback
RC1=1
RC2=2

So you can see that the returncodes are dutifully captured. If I reverse the testing order, the result stays the same.

So I think your basic approach is correct, and your test_XX_rc variables should hold the correct return codes.

BTW I would use

/apps/sasconfig/Lev1/SASApp/BatchServer/sasbatch.sh

for batch jobs, and I would never tolerate a return code ne 0. SAS programs that have a WARNING have failed, period.

Re: Running several SAS Program Files at once

KenMac — Wed, 22 Aug 2018 21:49:04 GMT

You recommend using 'sasbatch.sh' - any particular reason?

Re: Running several SAS Program Files at once

Kurt_Bremser — Thu, 23 Aug 2018 03:59:25 GMT

sasbatch.sh is specifically designed to run batch jobs. It is what scheduling through SMC uses.