RSUBMIT batches unresponsive

[bbpatterson]

Hi,

I have recently inherited a wealth of SAS programs previously established. Most of the programs are run via a nightly job in a master program using a "master macro" to run the programs in batch. However, sometimes a batch of programs does not produce results.
Below is an example one of several subsequent RSUBMIT batch run commands that are being called within the "Master Macro".
----------------------------------------------------------
RSUBMIT connectremote=First CONNECTWAIT=no;
%include 'Open.sas';run;
%include One.sas';run;
%include 'Two.sas';run;
%include 'Three.sas';run;
%include 'Four.sas';run;
%include 'Five.sas';run;
%include 'Six.sas';run;
%include 'Seven.sas';run;
%include 'Eight.sas';run;
%include 'Nine.sas';run;
%include 'Ten.sas';run;
%include 'Eleven.sas';run;
%include 'Twelve.sas';run;
%include 'Thirteen';run;
ENDRSUBMIT;
----------------------------------------------------------
There are probably 5-6 consecutive RSUBMIT statements in the Master Macro that call >10 programs each with a series of run commands.

My first guess would be that too many programs are being called in batch, causing a locking issue.

This First batch above gets the first 3 reports, but then subsequent reports Four-Thirteen do not produce results and the logs appear to bounce quickly through them, logging:

"NOTE: Table WORK.FIVE created, with 0 rows and 4 columns."
"NOTE: The data set WORK.FIVEDATA has 0 observations and 3 variables."
"NOTE: The SAS System stopped processing this step because of errors."
real time 0.00 seconds
cpu time 0.00 seconds

I have found the WAITFOR command, would this possibly help alleviate an overload and provide process control for each RSUBMIT batch to complete before the next one is processed?

Is there any additional block that I could put each of the run calls in to ensure these are processed prior to the next being ran?

Any help or suggestions would be greatly appreciated!

_b

[darrylovia]

i don't think the CONNECTWAIT option is causing your problems. Since the log snippet isn't producing an ERROR maybe your data doesn't have any rows which may be unexpected.

Is there any subsetting based on dates or anything like that when the data sets like WORK.FIVE is being produces?

[SASKiwi]

What happens if you run just one RSUBMIT batch and wait for it to finish? Does it work OK? If it does, then I think that interaction between batches (possibly processing the same datasets at the same time) when you do multiple RSUBMIT batches may be the problem.

Try CONNECTWAIT = YES to force all of your batches to run consecutively. Do the batches now work OK?

[bbpatterson]

Thanks for the suggestion. I will try CONNECTWAIT=yes and see if this helps.

So CONNECTWAIT=yes with each RSUBMIT will make the system wait until the completion of the RSUBMIT before stepping into any subsequent RSUBMIT or code in the Macro?

Thanks!

[SASKiwi]

That's correct. CONNECTWAIT = no means SAS does not wait for the RSUBMIT block to complete before running subsequent steps. This allows you to process in parallel as long as you are doing it in different remote sessions, which you appear to be doing.

[bbpatterson]

Thanks for your help. I am however still seeing failures of programs ran within the RSUBMIT blocks.

Does the SAS system fail subsequent programs when a preceding program fails?

If program Two.sas fails, then will Three-Six fail/dump out?

RSUBMIT connectremote=First CONNECTWAIT=yes;
%include 'Open.sas';run;
%include One.sas';run;
%include 'Two.sas';run;
%include 'Three.sas';run;
%include 'Four.sas';run;
%include 'Five.sas';run;
%include 'Six.sas';run;
ENDRSUBMIT;

[SASKiwi]

There are a number of SAS options that affect what SAS does if there is an error. These include NOSYNTAXCHECK and ERRORABEND. I suggest you check your default settings for these values (PROC OPTIONS) and check out the documentation for these.

It would be helpful if you could provide the SAS errors you are getting as well so we can give you a more informed response.