topic Re: Batch processing much slower than interactive session in SAS Programming

Batch processing much slower than interactive session

MB_Analyst — Mon, 24 Jan 2022 13:20:31 GMT

I have syntax that, when run in batch mode, is significantly slower than in an interactive session. I checked the output of proc options;run; and I'm not seeing any major differences. Steps that often bog down in batch mode are anything I/O, or proc sort. For example, the following code snippets takes a less than 3 minutes each during an interactive session, and 10+ minutes each in batch. Although the real time increases to around 10 minutes, the CPU time remains around ~5 seconds.

/* Example sql pull from server location */
proc sql;
    create table want as
    select
    a.*,
    b.*
    from have1 a
    left join have2 b
    on a.id = b.id
    order by a.id, a.date;
quit;

/* example data step with work library */
data want2;
    set want(where=(id ne .));
    by id date;
    if first.id then output;
run;

My guess is the difference could be due to a number of things, but wondering if anyone had thoughts on options to check? The code runs daily, and I plan to add FULLSTIMER to get a better sense of bottlenecks. Any ideas why batch would be double+ the time to run?

Edit to add in a difference in CONFIG locations:

Batch CONFIG:

CONFIG=( "C:\Program Files\SASHome\SASFoundation\9.4\sasv9.cfg" "C:\Program Files\SASHome\SASFoundation\9.4\nls\en\sasv9.cfg" )

Interactive CONFIG:

CONFIG=C:\Program Files\SASHome\SASFoundation\9.4\nls\en\sasv9.cfg

Re: Batch processing much slower than interactive session

Kurt_Bremser — Mon, 24 Jan 2022 12:54:02 GMT

Look at the setup of your WORK and UTILLOC in both environments; also post logs from both environments here after you activated FULLSTIMER.

Re: Batch processing much slower than interactive session

Ksharp — Mon, 24 Jan 2022 12:55:35 GMT

Do you have LOG of batch mode ? Like *.log

"D:\SASHome\SASFoundation\9.4\sas.exe" -nosplash -sysin "c:\temp\temp.sas" -log "c:\temp\temp#Y#m#d-#H-#M-#s.log" -logparm "rollover=auto" -set dsn "sashelp.air" -set age "'01jan1955'd"

Re: Batch processing much slower than interactive session

MB_Analyst — Mon, 24 Jan 2022 12:59:44 GMT

Should I check something for these beyond the options output?

Both batch and interactive mode options have:
UTILLOC=WORK

And the WORK is set to my local temp files (C:\Users\username~1\AppData\Local\Temp\SAS Temporary Files\_TDxxxx_)

Re: Batch processing much slower than interactive session

MB_Analyst — Mon, 24 Jan 2022 13:05:00 GMT

I do have the log being output. A scheduled task runs a "master file" that outputs in one location, and runs %include statements to run other code. The %include statements use proc printto to output individual .log files for each syntax file.

The bat file consists of the following:

"C:\Program Files\SASHome\SASFoundation\9.4\sas.exe" -SYSIN "C:\path\to\master\masterCode.sas" -log "C:\path\to\logs\%DATESTAMP%_masterLog.log" -noprint

Re: Batch processing much slower than interactive session

Ksharp — Mon, 24 Jan 2022 13:10:15 GMT

Then check your LOG file and could find out which part of code is the most wasting time .

Re: Batch processing much slower than interactive session

Kurt_Bremser — Mon, 24 Jan 2022 14:26:49 GMT

Then it may well be that your background (batch) processes have a lower priority than your foreground (interactive) ones. This makes sense on a desktop.

Re: Batch processing much slower than interactive session

Tom — Mon, 24 Jan 2022 14:34:28 GMT

Compare the CPU times between the two runs instead of just the REAL times. It could just be that the job is running with lower priority or when the machine is busier.

Are you running multiple SAS sessions at the same time? If they are all using the same disk for WORK then there will be a lot of disk contention in addition to needing to share the CPU.

Note if you are going to be running multiple SAS sessions are the same time then make sure to use the RSASUSER option when you start SAS. This will open the SASUSER libref in readonly mode. Then when you try to open a new interactive SAS session while another SAS session is running you still get access to your personalized settings, like window locations and key mappings.

Re: Batch processing much slower than interactive session

MB_Analyst — Mon, 24 Jan 2022 16:43:40 GMT

I will look into this. It could be promising because if I run the .bat manually (i.e. double clicking), it runs much faster than when the schedule tasks prompts the .bat.

I will change the priority based on this webpage's info, as I confirmed SAS is set at a lower priority when run from a batch file.