topic Re: how to run mutliple SAS files in batch mode with single program/ or simple way in SAS Programming

how to run mutliple SAS files in batch mode with single program/ or simple way

SASuserlot — Thu, 14 Oct 2021 21:51:46 GMT

I have multiple table programs in a folder that need to be run in batch mode. I am presently running them individually one at a time or with one Programming code where I Included the SAS tables file names that I need with %include and run. I would like to know is there a simple way to run this or automate it? The following image is an example to refer. Any help with this greatly appreciated. If you know any reference material please suggest me. Thanks

Re: how to run mutliple SAS files in batch mode with single program/ or simple way

SASKiwi — Thu, 14 Oct 2021 22:12:57 GMT

In a single SAS program you can do this:

%include "C:\Batchrun\prog1.sas";
%include "C:\Batchrun\prog2.sas";
%include "C:\Batchrun\prog3.sas";

This will run programs prog1, prog2 and prog3 in the order specified. You could do a wildcard %INCLUDE (*.sas) but that won't necessarily control the order the programs are run in.

Re: how to run mutliple SAS files in batch mode with single program/ or simple way

SASuserlot — Thu, 14 Oct 2021 22:54:07 GMT

Thanks for the response SASKiwi. I mentioned in my text that, I am presently running with %include only, But it won't create log or lst file for the programs. I also want to create the lst and log files.

Re: how to run mutliple SAS files in batch mode with single program/ or simple way

SASKiwi — Thu, 14 Oct 2021 23:48:45 GMT

To do that run SAS from the Windows command line: sas "MyProg.sas". Check this link.

Re: how to run mutliple SAS files in batch mode with single program/ or simple way

Patrick — Fri, 15 Oct 2021 00:36:00 GMT

Your question touches on two distinct areas:

1. How to schedule (automate) processes

2. How to structure (design) your jobs

Here some links for scheduling (there is much more out there):

https://blogs.sas.com/content/sgf/2013/08/14/four-ways-to-schedule-sas-tasks/

https://go.documentation.sas.com/doc/en/bicdc/9.4/scheduleug/titlepage.htm

In regards of job design

I would NEVER have fully qualified paths in individual jobs. Ideally define your libraries in metadata and/or create some %init macro where you define such paths.

Normally the folder structure within a "project" is meant to be stable but the path to the project folder can change. Often it's worth to define the root path to the project folder in such an %init_projectA() macro (or even the autoexec); something like %let projectA=<path>.

Then within your code you still can have a libname or filename like fiilename myfile "&projectA/<foldera>/<folderb>/myfile.txt";

Make sure that you store the %init_projectA() macro in a folder that's part of the SAS Autocall facility - or add the folder to this facility via extending the SASAUTOS option (in the usermods version of the .cfg or autoexec).

...above written a bit quickly and I'm sure there are good papers and blogs out there if you search a bit.

Re: how to run mutliple SAS files in batch mode with single program/ or simple way

yabwon — Fri, 15 Oct 2021 07:44:25 GMT

If your session has the XCMD option on, you could use this macro (read comments inside the macro for details and to adjust it for your task):

%macro runInParallel(filesLocation);

/* get the list of files */
data filesWithCodes;
  base = "&filesLocation.";
  length file $ 256 folderRef fileRef $ 8;

  folderRef = "_%sysfunc(datetime(), hex6.)0";

  rc=filename(folderRef, base);
  folderid=dopen(folderRef);

  N = dnum(folderId);
  do i=1 to N; drop i;
    file = dread(folderId, i);
    call symputX(cats("TEST_", i), file, "L");
    output; 
  end;

  rc = dclose(folderid);
  rc = filename(folderRef);

  call symputX("numberOfTests", N, "L");
  stop;
run;
proc print data = filesWithCodes;
run;

/* setup sas sessions */
%local SASROOT SASEXE SASWORK;
filename sasroot "!SASROOT";
%let SASROOT=%sysfunc(PATHNAME(sasroot));
filename sasroot;
%put *&SASROOT.*;
%let SASEXE=&SASROOT./sas;
%put *&SASEXE.*;
%let SASWORK=%sysfunc(GETOPTION(work));
%put *&SASWORK.*;

%local t;   
systask kill 
  %do t = 1 %to &numberOfTests.;
    sas&t. 
  %end;
wait;

/* run parallel jobs */
/* you can adjust the config file location yourself by editing `-config ""&SASROOT./sasv9.cfg""` */
/* you can adjust outputs and logs locations yourself */
%do t = 1 %to &numberOfTests.;
%local sasstat&t.;
systask command
"""&SASEXE.""
  -sysin ""&filesLocation./&&TEST_&t.""
  -print ""&filesLocation./&&TEST_&t...lst""
    -log ""&filesLocation./&&TEST_&t...log""
 -config ""&SASROOT./sasv9.cfg""
   -work ""&SASWORK.""
 -noterminal
 -rsasuser"
taskname=sas&t.
status=sasstat&t.
NOWAIT /* change it to WAIT if you dont need to run it in parallel */
;
%end;

waitfor _all_
  %do t = 1 %to &numberOfTests.;
    sas&t. 
  %end;
;

data _null_;
  put "NOTE: The End!";
run;

%mend runInParallel;

options NOQUOTELENMAX;
%runInParallel(C:\Users\bart\Desktop\abc)

All the best

Bart

Re: how to run mutliple SAS files in batch mode with single program/ or simple way

SASuserlot — Wed, 20 Oct 2021 13:07:15 GMT

Thank you so much . I tried this macro, It covers 80% or my requirement. However, there was an issue for me. This macro running all the files in batch mode even though they are not . SAS files. In my case its running the PNG, and even folders and creating the log file. I am not familiar with the system options, I tried controlling the SAS files but its not working. do you have any idea how to modify it to run only SAS files!

Re: how to run mutliple SAS files in batch mode with single program/ or simple way

yabwon — Wed, 20 Oct 2021 17:53:10 GMT

Change the data step from the macro to:

data filesWithCodes;
  base = "&filesLocation.";
  length file $ 256 folderRef fileRef $ 8;

  folderRef = "_%sysfunc(datetime(), hex6.)0";

  rc=filename(folderRef, base);
  folderid=dopen(folderRef);

  N = dnum(folderId);
  j = 0;
  do i=1 to N; drop i;
    file = dread(folderId, i);
    if upcase(scan(file, -1, ".")) = "SAS" then
      do;
        j+1;
        call symputX(cats("TEST_", j), file, "L");
        output; 
      end;
  end;

  rc = dclose(folderid);
  rc = filename(folderRef);

  call symputX("numberOfTests", j, "L");
  stop;
run;
proc print data = filesWithCodes;
run;

Bart

Re: how to run mutliple SAS files in batch mode with single program/ or simple way

SASuserlot — Thu, 21 Oct 2021 18:14:34 GMT

Perfect. It worked , Thanks. 🤗