DATA Step, Macro, Functions and more

Tracking the progress of a remote data step

Reply
N/A
Posts: 1

Tracking the progress of a remote data step

Is there any way to see which record is currently being processed in a data step that is inside rsubmit?

There is plenty ways of tracking the progress when we have a local data step, but I have no idea how to deal with remote data steps.

PROC Star
Posts: 7,366

Tracking the progress of a remote data step

Depending upon how closely you want to track, you may only need:

http://support.sas.com/documentation/cdl/en/connref/61908/HTML/default/viewer.htm#a002590489.htm

Trusted Advisor
Posts: 1,300

Re: Tracking the progress of a remote data step

Also, you can pass an altlog option to your sascmd statement in your signon statement.

Such as:

signon task1 sascmd="/usr/local/SAS/SASFoundation/9.2/sas -altlog /mylogs/mylog.log" wait=no;

rdisplay is also useful as long as you are running you program interactivly.

Super User
Posts: 3,115

Re: Tracking the progress of a remote data step

You can also put comments in your log from within your DATA step like so:

if mod(_n_, 100000) = 0 then put 'Processing Row: ' _n_;

This will write a comment for every 100,000 rows processed. The difficulty with remote processing and this applies to batch SAS jobs as well is that the log is buffered and only updated when "sufficient" lines have been written.

Trusted Advisor
Posts: 1,300

Tracking the progress of a remote data step

SASKiwi makes a good point and references another topic that I had actually posted:

http://communities.sas.com/message/103760#103760

I have still not had an opportunity to look into using the write=immediate option for logparm.


FriedEgg wrote:

I am not currently able to test but I know in my current enviornment the following will not work:

Curent Enviornment: LOGPARM=WRITE=BUFFERED ROLLOVER=NONE OPEN=REPLACE

data _null_;

  do i=1 to 10000000;

   call ranuni(1234);

   if mod(i,50000)=0 then putlog i=;

end;

run;

If the write=immediate option doesn't resolve the issue my next thought is to call sysexec and cat a note onto the end of the logfile (if I even can incase there is a lock on the file held by sas while using write=buffered?).

data _null_;

  do i=1 to 10**6;

   if mod(i,100000)=1 then call sysexec('x echo ' || put(i,comma.) || ' > /mylogs/tracker.dat'

  end;

run;

Super User
Posts: 3,115

Tracking the progress of a remote data step

Here's a reference to the SAS doc covering LOGPARM as discussed by FriedEgg:

http://support.sas.com/documentation/cdl/en/lrcon/62955/HTML/default/viewer.htm#a000998454.htm

I haven't tried the WRITE=IMMEDIATE option either but would be interested in feedback from anyone who has tried it.

Ask a Question
Discussion stats
  • 5 replies
  • 510 views
  • 0 likes
  • 4 in conversation