SAS Email if Errors

[Quentin]

HI Jaap,

In the case of log checking, many folks have been using customized %logcheck() macros since long before the more recent SAS supplied logging model.  Thus this qualifies as "reuse," particularly when the macros are part of a shared enterprise macro library.

I was inspired to work on one after seeing:

http://www.nesug.org/proceedings/nesug01/cc/cc4008.pdf

As pointed out in the paper, developing your own log scanning algorithm allows you to decide what is worthy of considering a bad log message.   There are many notes that people treat as errors.  The authors argue that the most conservative approach for log scanning is to provide a white list of accepted log NOTES, and treat any other message as an ERROR:. 

As Kurt points out (and I did not mention), this customized log scanning/emailing does not preclude one from using the error notification features present in the scheduler or other parts of the SAS environment, and often they complement each other.  The pograms I described are all scheduled through LSF using SMC, and the job metadata defines whether SAS will send job completion emails, etc.

[jakarman]

Yes quentin, I know you have a different background as to the one I have.
I am seeing a lot happening with assumptions of people not really cooperative with their IT staff.

As have being also part of IT, I am seeing that form an other perspective. So we are discussing again.   
Nesug has stopped papers can be found at lexjanssen. This one is 2001 based section codes-corner.

Year 2001, I'll go back to 1980's. Scheduling was to be introduced yet at IT (ca-7 ucc7 ctl-d opc a lot of the same).

A lot of resistance by the operators ("bridge") and the operations-support department. They did it always manually and were thinking doing it better and cheaper. Do a guess on the outcome.     

So we are 30 years later now and what kind of discussions we are in?  The planning and monitoring done by business users at the same level as those old days by operators. 

[Quentin]

Thanks Jaap, but you are mistaken to think that the approach I described briefly is necessarily a personal approach developed/owned by individual users/operators rather than an enterprise solution managed by SAS admins / IT.

Use of SAS Autocall facility is an excellent tool for enterprise-wide standardization, and provides flexibility (when desired) to allow standardization at various levels (enterprise/server/project/etc.).

An enterprise %logcheck() macro provides opportunities for the same error detection algorithm to be used in a variety of environments (old fashioned interactive DM SAS, batch jobs, DI jobs, EG... ).  The log scanning can happen in "real time", or after-the-fact (given a directory full of logs, scan them all).

Not too surprisingly, when I look into the code for a DIS job that has been deployed for scheduling via SMC, what do I see at the end in the code generated by SAS?  A macro for checking return codes and emailing:

%macro etls_jobRCChk; 
   %if (&job_rc ge 5) %then
   %do; 
      %macro etls_sendEmail(address=, message=); 
      
         filename sendMail email "&address" subject='ETL Process Status'; 
      
         %local etls_syntaxcheck; 
         %let etls_syntaxcheck = %sysfunc(getoption(syntaxcheck)); 
         /* Turn off syntaxcheck option to perform following steps  */ 
         options nosyntaxcheck;
      
         data _null_; 
            file sendMail; 
            dttm = put(datetime(),nldatm.); 
            put dttm "&message."; 
         run; 
      
         /* Reset syntaxcheck option to previous setting  */ 
         options &etls_syntaxcheck; 
      %mend etls_sendEmail; 
      %etls_sendEmail 
         (address = me@myhouse.com, 
          Message = Error); 
   %end; 
%mend etls_jobRCChk; 
%etls_jobRCChk; 

I would prefer it if SAS provided these macros in the autocall library, to make it easier for an enterprise to customize these routines according to their own needs. 

[jakarman]

Well Quentin we are that close together and still living in different worlds.
I would like to support autocall-macros when needed and if needed that is not a SAS-institute task, there we are different, but your internal IT-staff support. I did this kind of stuff and I know the challenges around that. I hope you understand hosting and multi-tenancy. With that you can see what is to be done around that.
Your sample with DI that is given I would advice not to use. Finding some different way to solve requirements.
What is the reason? (rhetoric)

I am seeing something that is environment dependent. Mail-addresses, User-ids, passwords, physical file names, DBMS connection should not get part of the generated code. That is some classic coding standards to get that reliable in a release-management. Adjusting code in an operational production environment is just "not done".
I have seen SAS has documented that as the solutions to solve those kind of issues. That is as bad way and should be killed.  

There are policies guidelines to follow coming in at an other level. I am sorry that the IT world is little bit more as just SAS.  

[Quentin]

Well, at least we can agree that there are several deficiencies in the generated %etls_jobRCchk macro that I posted, starting with nesting the defintion of %etls_sendEmail within it. : )