Writing logs from rsubmit separately

[cfrey73]

Hi!

I am working with rsubmits in SAS EG to parallelize my code, submitting batches of 20 at a time (see screenshot). Currently the whole log just goes into one file, I am wondering if there is a way to have each remote session write a log to a different file so that the log doesn't get extremely long. 

Thank you

[cfrey73]

[jimbarbour]

That's an extremely useful technique, having each sub-process write to a separate log.  The way I do it is to set up an ALTLOG.  Enterprise Guide can be a bit of a pain with RSUBMIT.  It's hard to read the output in the regular log -- and then nearly impossible in the Project Log.

What I do is first set up a macro variable with the commands I want:

%LET		Cmd_String				=	!sascmdv -logparm 'write=immediate rollover=session'  -memsize 8G 
-altlog &Log_Path\sub_process_logs\&Pgm._#Y-#m-#d_#H-#M-#s_#p.log;

SAS uses the value of the cmd option to launch RSUBMITed jobs. The existing value of the SAS cmd option can be brought in via !sascmdv.  All I'm doing is appending some parameters to the end of the existing cmd string, one of which is an ALTLOG parameter.  The #Y #m and such are log directives that embed a date time stamp and the Process ID in the log file name.

Then I set the SAS cmd option to the value of the string I just created in the macro variable:

OPTION		SASCMD					=	"&Cmd_String";

Then I execute my RSUBMIT just as I normally would:

RSUBMIT	PROCESS=&Process_Name		WAIT=NO	PERSIST=NO;
	%INC	"&Pgm_Path\&Pgm..sas";
ENDRSUBMIT;

During execution, I can monitor the jobs via the sub-process logs as they execute because I specified WRITE=IMMEDIATE in my logparm.  And of course afterwards, it's a lot easier to work with the logs since each sub-process has it's own log.  The date time stamps allow me to keep runs separate even if I need to move, copy, or archive the jobs.  During execution the Process ID helps me look at my Task Manager to see which processes are still running, how much CPU they're using, etc.

Jim