Changing project's behavior at runtime

[lc_isp]

    Hi all,

I'm working with a SAS EG project which needs to "continue running in case of errors": to configure that (non-default) behavior there is a menu choice (Properties > Project properties > Code submission > Action to take on errors ...: Continue execution).

Configuring the project this way, my first step makes the project to solve a problem I'm having with a library so that's ok.

In the following steps tho, I would like to be able to revert project's behavior to its default (so to stop, if it gets an error): I was wondering if there was any way, e.g. a system variable one can (re)set at runtime, for this to happen (e.g. %let &sysprjerrcodesubmit = "stop", or "0" or such), before to leave step 1.

All that 'cause the project will be executed via batch scheduler, so no human intervention will be there, to manually revert the behavior (and other reasons too).

[Patrick]

Given this is a project property I highly doubt that you can change this on a task level. There might be coding options instead to have SAS continue to execute after an Error.

If you say: Running in batch do you mean EG scheduling or "real" batch processing via some other scheduler? 

If it's not just EG scheduling then many schedulers allow to define what should happen in case a task node fails - which can also be to continue running.

Ideally you're writing code that never throws errors or warnings. May-be share your code or at least describe in detail which error condition could occur where you still want to continue processing so we can propose some check logic instead that doesn't throw an Error (like fileexist or similar).

[lc_isp]

Hi Patrick, and thanks for your hints too.

As for Quentin's hints, SAS EG nowadays have ways to decide, at task-level, what to do when an error condition arises (this while still keeping the global setting in action, which is good).

I thought there was a way to "switch" such "global flag" but it's ok to operate this way too (it's a setup, after all): both ways solve my problems.

About the project and the reason why some parts are expected to raise errors: I believe we're having a "not properly configured" library access, for which (and differently than other libraries we're accessing already) a library I'm accessing to automate some data-extraction results unassigned when I open my project.

As far as I'm working interactively it's not a problem, as I can re-assign the library in a moment (and always succesfully) but, as my main work is to automate processes, I was wondering what happened if the project was launched by SAS EG Scheduler (or even by SAS Management Console, which I've no access to, at the moment, but my hopes are toward it, for the future): whichever "scheduler" will be involved, no human intervention is possible, if the task runs "by its own" (scheduler) at 1am everyday.

Thus, I dug the net, to try solving such case (and ok, I totally agree that the real solution is to ask for a proper library access configuration... but, sometimes, such things are terribly hard/time consuming to obtain: an "alternative solution" could help anyway).

So I put together a project step where the first task checks for the library assignment, if ok it sets a variable (&libassigned) to 1, so to take profit of the conditional execution.

If the next task gets that variable as 0, EG starts a (re)assignment task (and here starts the weird): I noticed that, even when the (re)assignment works, the following tasks fails anyway (with "library not assigned" error), like there was "something wrong" with metadata. Searching more, I solved the problem by using proc metalib, which I'm not authorized to but, even when it fails, it seems it successfully sync metadata, so that the following steps works properly: that's why I need, in this project-step only, the execution to keep going even when a task (proc metalib) fails.

By using Quentin's hints, configuring the "go on" behavior to a few tasks, I can leave the rest of the project "clean" (considering that behavior only) to my colleagues, which are used that the project  stops if errors arise.

With time, I believe I'll manage to achieve a better/proper access setup to that library too but, for the moment, my "patch" is working.

Best,

[Patrick]

@lc_isp So if I understand right then you've got a library in SAS metadata where you're not sure if it's pre-assigned or not. If so then you could first run a SQL query against dictionary.libnames to check if the expected libref exists.

If the libref does not exist then you could assign it via a libname statement that uses the existing metadata library definition. 

%let libexist_flg=0;
proc sql;
  select count(*)=1 into :libexist_flg trimmed
  from dictionary.libnames
  where libname='MYLIB'
  ;
quit;

%if libexist_flg=0 %then
  %do;
    libname mylib meta SASLibrary?@libref='mylib' metaout=data;
  %end

[lc_isp]

@Patrick I'm nearly sure, from SAS behavior, the library I'm accessing is not pre-assigned: sometimes I had no problems to "just access it" (e.g. running a program which read a table), but that's possibly 'cause I accessed it some hours before: 90% of the times I "just run a program" which tried accessing it, SAS gave me back an "ERROR: Libref <libname> is not assigned.".

To have the access modified, it's probably "harder" (time consuming, with uncertain results) than implementing the check-and-retry mechanism you also depicted in your code.

About your code: I tried modifying it, to work with a parameter (&lib)

%let libexist_flg=0;
%let lib=XYZ123;

proc sql;
  select count(*)=1 into :libexist_flg trimmed
  from dictionary.libnames
  where libname="&lib."
  ;
quit;

%if libexist_flg=0 %then
  %do;
    libname &lib. meta SASLibrary?@libref="&lib." metaout=data;
  %end

but SAS seems having a problem with the SASLibrary?@libref="&lib" part, and I still don't know SAS programming enough (zero courses) to see where the error is:

44       !                          SASLibrary?@libref="&lib." metaout=data;
                                    __________  ______
                                    23          22
ERROR 23-7: Invalid value for the SASLIBRARY option.

ERROR 22-7: Invalid option name LIBREF.

44       ! SASLibrary?@libref="&lib." metaout=data;
                                              _
                                              23
ERROR 23-7: Invalid value for the ? option.

I probably misunderstood your code about some parameter.

[Patrick]

@lc_isp  Below code works in my environment (tested).

%macro assign_lib(lib);
  %let lib=%upcase(&lib);

  %local libexist_flg;
  %let libexist_flg=0;
  proc sql noprint;
    select count(*)>1 into :libexist_flg trimmed
    from dictionary.libnames
    where libname=%upcase("&lib.")
    ;
  quit;
  %put &=libexist_flg;

  %if &libexist_flg=0 %then
    %do;
      libname &lib. meta liburi="SASLibrary?@libref='&lib.'" metaout=data;
    %end;
%mend;
%assign_lib(mylib);

In above code the macro call expects a library in SAS metadata defined with a libref mylib. 

[lc_isp]

@Patrick thanks again for the code and explanations (and your testing time).

I changed the code a little, and tested it: it's working, and without the need of proc metalib / error management. I'll use it more, in the project I'm working on, to see if it needs further changes.

I'm showing the changes, below, if someone likes to use them (last part, about proc contents, is optional, just to test libref have been really assigned and the library is available to use)

OPTIONS mprint mprintnest mlogic mlogicnest symbolgen syntaxcheck autocorrect fullstimer source stimer threads;	/* DEBUG: _VERBOSE_ */
/*OPTIONS nonotes nomprint nomprintnest nomlogic nomlogicnest nosymbolgen nosyntaxcheck noautocorrect nofullstimer nosource nostimer threads;*/	/* INFO: SHORT */

%global libexist;
%global mylib;

%let mylib=gn777A;																/* lower-case chars are on purpose */

%macro assign_lib(lib);
	%let lib=%upcase(&lib);

	%put &=lib;

	/*%local libexist;*/
	%let libexist=0;
	proc sql noprint;
		select count(*)>1 into :libexist trimmed
		from dictionary.libnames
		where libname=%upcase("&lib.")
		;
	quit;
	%put &=libexist;

	%if &libexist=0 %then
		%do;
			libname &lib. meta liburi="SASLibrary?@libref='&lib.'" metaout=data;
		%end;
%mend;

%put &=mylib;

%assign_lib(&mylib.);

%put &=libexist;

/* just to check if libref was finally assigned */
%if &libexist. ne 0 %then %do;
	proc contents data=&mylib.._ALL_ nods short;
	run;
%end;
%else %do;
	%put &mylib. NOT assigned;
%end;

[Patrick]

@lc_isp If you make &libexist global then you need to set it to zero before calling the macro as else the macro will only in the first call return a valid result. One of the reasons to use a macro is its reusability meaning you only need to define it once per session but then can call it many times. For that reason I fail to see how your changes would be an improvement.

[lc_isp]

@Patrick my SAS programming knowledge is sure lesser than yours: I thought the 

%let libexist=0;

in macro's body, worked that way. I mean: if I remove the "%local libexist" from the macro, it should refer to the global one, right?

[Quentin]

---

@lc_isp wrote:

@Patrick my SAS programming knowledge is sure lesser than yours: I thought the 

%let libexist=0;

in macro's body, worked that way. I mean: if I remove the "%local libexist" from the macro, it should refer to the global one, right?

---

Yes, if there is a global macro variable LIBEXIST and your remove the %LOCAL statement, then a reference to LIBEXIST in the macro will point to the global version.  But it's not clear to me if you are gaining any benefit from having a global macro variable.  Note that you seem to be using it after your macro call, to see if the code worked.  But if the libref didn't exist before the macro call, and the macro successfully generates the LIBNAME statement, LIBEXIST will still be 0 after the macro call.

I noticed there is a LIBREF function which returns 0 if a libref is not assigned.  I wonder if you could accomplish your goal with (untested):

%if (%sysfunc(libref(gn777A)) ne 0) %then %do ;
  libname gn777A meta liburi="SASLibrary?@libref='gn777A'" metaout=data ;
%end ;

Then if you want to check that the library exists after running the code, you could add an assertion: 

%if NOT (%sysfunc(libref(gn777A)) = 0) %then %do ;
  %put ERROR: libref gn777A could not be assigned! ;
%end ;

[lc_isp]

The reason why I'm using a global, instead of a local value, is to be able to test it, if I need, into other project steps without the need to re-call the testing macro (of course it can be done, if one needs or have doubts the library is still assigned, at that further point).

About @Patrick code (and aside of my little modification), as far as I tested it, it worked very well, as well as I'm replacing my original one (the one where I had the problem which made me to open this thread) with his one.

In my previous tests, with my code, I was seeing that, even when the library have been assigned by using

libname &library_name. META LIBRARY="&library_name." metauser=&_METAUSER.;

when I re-tested it a little later, the project gave the error like it wasn't assigned. But, when I gone to see the library interactively, right-clicking on it, the contextual menu gave me the unassign item, which seems not congruent with the "library not assigned" error my program received.

That's why I thought to use proc metalib

proc metalib;
	omr (library="&library_name.");
run;

as, even when it fails ('cause of my auth lack), it seems syncing metadata: result is the next test works

proc contents data=&lib..&tab.;
run;

@Patrick 's code solved the problem at the root, it seems as, after the

libname &lib. meta liburi="SASLibrary?@libref='&lib.'" metaout=data;

engaged if &libexist is 0 only, further tests correctly access the library, all the solution into a single macro, also.

I could then figure how to modify it, to test multiple libraries but, for a single one, it's working already.

About the "assertion" part: it could be more correct to have another test, after the above libname... assignment, so to be sure the library is assigned, after that call (I know I'm assuming it is, right now) to end with librassigned = 1 to one side or the error message (plus libassigned = 0) to the other, you're right on that.

[Quentin · Accepted solution]

I haven't tested, but according to this paper by the great @CaseySmith  https://support.sas.com/resources/papers/proceedings17/SAS0562-2017.pdf, in addition to setting this option at the project-level you can set it for an individual item in your EG flow.  So sounds like you can leave the project configuration as the default (stop on error) and then change the option for the first item in the flow to "continue execution."

[lc_isp]

TYVM Quentin: your hint totally solved my problem! 👍

I did all the testing, creating errors on purpose, and the (single) tasks configured to "keep going" left the process running through, which is exactly what I needed as, sometimes, even errors are wanted/needed, to check some conditions or about other purposes: this way we avoid the whole project is stopped, if a (wanted) error happens.

As I needed such behavior into a limited part of my project only, all other tasks are still free to use the global setting (so, to stop if the project falls in error, which is the wanted behavior).

cheers!