topic Re: SAS help - Batch Mode - Running macros in loop with %abort statement in SAS Programming

SAS help - Batch Mode - Running macros in loop with %abort statement

Ramesh_165 — Fri, 10 Jul 2020 17:23:11 GMT

Hi There,

I’m looking for a solution to use %abort statement in one of the many sub-macros in a batch environment with the below need.

I’m running a macro (lets say, Main_Macro) with quite a few submacros. One of the sub-macro is quit_on_error which has an %abort (without argument) statement. The quit_on_error macro is called from multiple places of the sub-macros to quit the process if there is any error in any of the macros.
I want to run the Main_Macro in loop, lets say, for each US State. When a particular run (particular state) of the Main_Macro fails (%abort ed by quit_on_error), I want other states to continue. I achieved this using CALL EXECUTE statement to call the Main_Macro.

Now the problem is I tested the above two points in “interactive” mode, and it was working fine. Even though a particular run of the Main_Macro failed, the other runs were attempted to process.

But when I put this code in “batch” mode, when a particular run of the Main_Macro failed all the subsequent process are suspended.

I believe this is due to the behavior of %abort statement between interactive and batch modes.

https://documentation.sas.com/?docsetId=mcrolref&docsetTarget=p0f7j2zr6z71nqn1fpefnmulzazf.htm&docsetVersion=9.4&locale=en

Is there a way to have the rest of the runs to continue process even though the previous run failed in batch mode? Any help or insights are much appreciated.

Thanks in advance!

Ramesh

Re: SAS help - Batch Mode - Running macros in loop with %abort statement

LeonidBatkhan — Fri, 10 Jul 2020 19:02:13 GMT

Hi Ramesh_165,

Instead of %abort statement try using %return statement which causes normal termination of the currently executing macro.

I also posted my reply in the CALL EXECUTE made easy for SAS data-driven programming blog post.

Best regards,

Re: SAS help - Batch Mode - Running macros in loop with %abort statement

Ramesh_165 — Fri, 10 Jul 2020 20:41:14 GMT

Hello Leonid,

Thanks for your response!

The reason I'm using %abort instead of %return statement is because I don't want the subsequent macro calls within the Main_Macro to execute once the QuitOnError macro is called.

Please see the below example where the Main_Macro has n-number of macros, and once the QuitOnError macro is executed from any of the macros, no subsequent statements or macros should execute. In this case I have coded, both %return and %abort statement conditionally to show the execution difference.

OPTIONS NOMPRINT NOMLOGIC NOSYMBOLGEN NONOTES;

%Macro QuitOnError(abortOrReturn);

%put &abortOrReturn. in QuitOnError macro;

%if &abortOrReturn = abort %then

%abort;

%else %if &abortOrReturn = return %then

%return;

%Mend;

%Macro macro_n(iteration);

%put &iteration. - Macron successful;

PROC SQL;

SELECT *

FROM SASHELP.CLASS;

QUIT;

%put &iteration. - Class Table count = &sqlobs.;

%Mend;

%Macro Macro_4(iteration);

%put &iteration. - Macro4 successful;

PROC SQL;

SELECT *

FROM SASHELP.CLASS;

QUIT;

%put &iteration. - Class Table count = &sqlobs.;

%Mend;

%Macro Macro_3(iteration);

%put &iteration. - Macro3 Errored;

%put &iteration. - calling QuitOnError from Macro3;

%if &iteration = 2 %then

%QuitOnError(abort);

%else

%QuitOnError(return);

PROC SQL;

SELECT *

FROM SASHELP.CLASS;

QUIT;

%put &iteration. - Class Table count = &sqlobs.;

%Mend;

%Macro Macro_2(iteration);

%put &iteration. - Macro2 started;

PROC SQL;

SELECT *

FROM SASHELP.CLASS;

QUIT;

%put &iteration. - Class Table count = &sqlobs.;

%Mend;

%Macro Macro_1(iteration);

%put &iteration. - Macro1 successful;

PROC SQL;

SELECT *

FROM SASHELP.CLASS;

QUIT;

%put &iteration. - Class Table count = &sqlobs.;

%Mend;

%Macro Main_Macro(iteration);

%Macro_1(&iteration);

%Macro_2(&iteration);

%Macro_3(&iteration);

%Macro_4(&iteration);

%Macro_n(&iteration);

%Mend;

DATA _NULL_;

DO i = 1 TO 3;

CALL EXECUTE('%NRSTR(%Main_Macro('||STRIP(i)||'))');

END;

RUN;

Put statement Log outputs of the above code:

1 + %Main_Macro(1)

1 - Macro1 successful

1 - Class Table count = 19

1 - Macro2 started

1 - Class Table count = 19

1 - Macro3 Errored

1 - calling QuitOnError from Macro3

return in QuitOnError macro

1 - Class Table count = 19

1 - Macro4 successful

1 - Class Table count = 19

1 - Macron successful

1 - Class Table count = 19

---------------------------------------------------------------

2 + %Main_Macro(2)

2 - Macro1 successful

2 - Class Table count = 19

2 - Macro2 started

2 - Class Table count = 19

2 - Macro3 Errored

2 - calling QuitOnError from Macro3

abort in QuitOnError macro

ERROR: Execution terminated by an %ABORT statement.

----------------------------------------------------------------

3 + %Main_Macro(3)

3 - Macro1 successful

3 - Class Table count = 19

3 - Macro2 started

3 - Class Table count = 19

3 - Macro3 Errored

3 - calling QuitOnError from Macro3

return in QuitOnError macro

3 - Class Table count = 19

3 - Macro4 successful

3 - Class Table count = 19

3 - Macron successful

3 - Class Table count = 19

-----------------------------------------------------------

Based on the log, I'm expecting the 2nd iteration behavior in my process, so I'm using %abort statement and not %return statement.

Though the 2nd iteration is aborted, it is running the 3rd iteration in interactive mode.

But in the batch mode, the process works fine until 2nd iteration. 3rd iteration starts fine, but all the observations are 0. I think it is because the OBS option is set 0 when %abort statement is executed in batch mode.

Below is log of the same code from batch mode:

1 + %Main_Macro(1)

1 - Macro1 successful

1 - Class Table count = 19

1 - Macro2 started

1 - Class Table count = 19

1 - Macro3 Errored

1 - calling QuitOnError from Macro3

return in QuitOnError macro

1 - Class Table count = 19

1 - Macro4 successful

1 - Class Table count = 19

1 - Macron successful

1 - Class Table count = 19

-----------------------------------------------------------

2 + %Main_Macro(2)

2 - Macro1 successful

2 - Class Table count = 19

2 - Macro2 started

2 - Class Table count = 19

2 - Macro3 Errored

2 - calling QuitOnError from Macro3

abort in QuitOnError macro

ERROR: Execution terminated by an %ABORT statement.

-----------------------------------------------------------

3 + %Main_Macro(3)

3 - Macro1 successful

3 - Class Table count = 0

3 - Macro2 started

3 - Class Table count = 0

3 - Macro3 Errored

3 - calling QuitOnError from Macro3

return in QuitOnError macro

3 - Class Table count = 0

3 - Macro4 successful

3 - Class Table count = 0

3 - Macron successful

3 - Class Table count = 0

-----------------------------------------------------------

Please note that the Main_Macro was written some back this way to use %abort statement. Now I’m just trying to run it in loop without not making much changes to code. There might be some elegant ways to check for errors at the start of each macro and go to the end if there were any errors.

Now I’m trying to see if there is a possibility at all to achieve the need without making the big change as there close to couple hundred macros within the Main_Macro.

Thanks again for reading this detailed note, and taking sometime to think about solution.

Ramesh

Re: SAS help - Batch Mode - Running macros in loop with %abort statement

Tom — Fri, 10 Jul 2020 22:42:19 GMT

Right. Don't do that.

If you want to run a series of jobs then run them as a series of jobs.

sas program1
sas program2
sas program3

Re: SAS help - Batch Mode - Running macros in loop with %abort statement

SASKiwi — Fri, 10 Jul 2020 22:53:50 GMT

Have you checked out the SAS option ERRORABEND? If this option is set then your batch program will abort when there is an error anywhere in your SAS job. There is no need to add checking steps all over your code in that case. I'm a fan of keeping coding practices simple and find the SAS options SYNTAXCHECK and ERRORABEND add all the control I need.

Re: SAS help - Batch Mode - Running macros in loop with %abort statement

s_lassen — Mon, 13 Jul 2020 07:27:12 GMT

What I normally do when wanting to break out of a program, but not end the SAS session, is to use the %ABORT CANCEL FILE statement; that only terminates the %INCLUDE file it is called from.

In your case, you could make a SAS program instead of your main macro, e.g. Main.SAS:

     %Macro_1(&iteration);
     %Macro_2(&iteration);
     %Macro_3(&iteration);
     %Macro_4(&iteration);
     %Macro_n(&iteration);

In your QUIT_ON_ERROR macro, you use %ABORT CANCEL FILE.

Your datastep could then be rewritten as:

DATA _NULL_;
     DO i = 1 TO 3;
           CALL EXECUTE(cats('%NRSTR(%let Iteration=',i,';%include Main.SAS;)'));
     END;
RUN;

Re: SAS help - Batch Mode - Running macros in loop with %abort statement

Ramesh_165 — Mon, 13 Jul 2020 18:31:27 GMT

Thanks for your response.

ERRORABEND option is set to NOERRORABEND. But as per the documentation, but the options would set OBS=0 when there is an error. So in my case, changing that may not help as I want the next iteration of the Main_Macro to run normally by just aborting the errored iteration.

Re: SAS help - Batch Mode - Running macros in loop with %abort statement

Ramesh_165 — Mon, 13 Jul 2020 18:42:08 GMT

The number of jobs is going to be varying. Thats why I'm controlling and calling Main_Macro within a datastep. As of now I have to run around 400+ iterations (this could go up to 2000+) of Main_Macro, and I have split them into 10 jobs. Still each job has to run 40+ iterations of Main_Macro. The problem is, in any of those jobs, if any iteration of the Main_Macro fails, the rest of the iterations of Main_Macro don't work. I believe SAS goes into a state where it won't process anything after that. I tried printing the options from dictionary.options table after the error, but no options are printed on the log. So either, SAS options are completely removed, or SAS is in a state where it can't process any datasets.

Re: SAS help - Batch Mode - Running macros in loop with %abort statement

Tom — Mon, 13 Jul 2020 18:50:28 GMT

I recommend running each separate job in its own session. Do you have SAS/Connect licensed? If so then it is pretty easy.

For example here is step I used to call a macro named EXTRACT_SUB in a separate SAS session. If it crashes it does not impact the current session.

*----------------------------------------------------------------------------;
* Create remote session and submit extract call ;
*----------------------------------------------------------------------------;
signon X sascmd="!sascmd -altlog '&outdir/&study/extract.log'";
%syslput study=&study /remote=X;
%syslput outdir=&outdir /remote=X;
rsubmit x;
  libname out "&outdir/&study";
  options insert=(sasautos="%gdir(macros)") compress=yes errorabend;
  %extract_sub(&study,&outdir);
  libname _all_ clear;
endrsubmit;
signoff x;

Re: SAS help - Batch Mode - Running macros in loop with %abort statement

Ramesh_165 — Mon, 13 Jul 2020 19:01:15 GMT

Thanks for your proposal. Based on the documentation, for batch jobs with %Abort cancel option would SAS program and SAS system.

CANCEL <FILE>

causes the cancellation of the current submitted statements. The results depend on the method of operation.

If the method of operation is batch mode and noninteractive mode, use the CANCEL option to do the following:

The entire SAS program and SAS system are terminated.
The error message is written to the SAS log

I tested the approach you have mentioned above and faced two issues.

1. The value of iteration variable is always 3 (max. loop number) for all the iterations of the program.

2. If the program encounters %abort cancel in the middle of iterations, for example 2nd iteration, it didn't execute the 3rd iteration of the program.

Please let me know if I'm missing something. Thanks again!

Re: SAS help - Batch Mode - Running macros in loop with %abort statement

Ramesh_165 — Tue, 20 Oct 2020 22:55:29 GMT

One of the difference between batch and interactive mode is what happens when SAS encounters an error. When an error occurs in interactive SAS, only the step that produces the error stops processing. In batch, an error causes SAS to go into syntax-check mode. This means that the system option OBS= is set to 0 and all subsequent steps are compiled only and not executed.

Setting the NOSYNTAXCHECK system option will have batch mode act like interactive mode.

options nosyntaxcheck;

Adding this NOSYNTAXCHECK option solved the above mentioned problem.

@all. - Thank you so much for your time.