Unable to read logfile saved in a folder in a data set.

[mohammad_umair]

Hi,

I am trying to capture the status of jobs and their exit code into a Table.

I'm saving the log in a folder using proc printto and then reading the log into a dataset for finding required values.

I'm using sas program step in my SAS Studio Flows for execution. 

When there is no error in the flow, my code executes successfully and I'm able to capture the details.

However, when I add an error in the flow, to test whether the details are being captured or not, I get errors.

Logfile is not being read by the program and no value for ERROR_CODE and ERROR are being saved.

How can I ensure that the logfile is read? I added a delay of 15sec and 60sec but its still unable to read log.

Code:

data _null_;
call sleep(60,1);
run;
options symbolgen mprint mlogic;

%macro m_upd_job_status(m_full_name=, m_job_name=,m_startdt=,m_end_dt=);
FILENAME REFFILE FILESRVC FOLDERPATH='/Project/Test/LOGS'  FILENAME="&m_full_name..txt";

%let Job_status=Completed;
%let ERROR_CODE= ;
%let ERROR_DESC= ;

data test;
    infile REFFILE truncover;
    input log_line $400.;
    I=index((log_line), 'SYSCC=');
run;

data _null_;
    infile REFFILE truncover;
    input log_line $400.;
    if index((log_line), 'SYSCC=') =1 then call symputx('ERROR_CODE', STRIP(SUBSTR(log_line,8)));
if index((log_line), 'ERROR ') > 0 or index((log_line), 'ERROR:') > 0  then call symputx('ERROR_DESC', STRIP(log_line));
run;

%if %sysevalf(%superq(ERROR_CODE) > 0) %then %do;
    %let Job_status = Failed;
%end;

%put &ERROR_CODE.;
%put &ERROR_DESC.;
%put &Job_status.;

proc sql;
    update LIB.AUDIT_TBL SET STATUS="&Job_status."
    where JOB_NAME = "&jobnm.";
    update LIB.AUDIT_TBL SET EXIT_CODE=&ERROR_CODE.
    where JOB_NAME = "&jobnm.";
    update LIB.AUDIT_TBL SET START_DTTM="&logdt."dt
    where JOB_NAME = "&jobnm.";
      update LIB.AUDIT_TBL SET END_DTTM="&endt."dt
    where JOB_NAME = "&jobnm.";
quit;
%mend m_upd_job_status;

%m_upd_job_status(m_full_name=&full_job. ,m_job_name=&jobnm.,m_startdt=&logdt.,m_end_dt=&endt.);

Log:

2032   data _null_;
2033   call sleep(60,1);
2034   run;
NOTE: DATA statement used (Total process time):
      real time           0.00 seconds
      cpu time            0.00 seconds
     
2035   options symbolgen mprint mlogic;
2036  
2037   %macro m_upd_job_status(m_full_name=, m_job_name=,m_startdt=,m_end_dt=);
2038   FILENAME REFFILE FILESRVC FOLDERPATH='/Project/Test/LOGS'  FILENAME="&m_full_name..txt";
2039  
2040   %let Job_status=Completed;
2041   %let ERROR_CODE= ;
2042   %let ERROR_DESC= ;
2043  
2044   data test;
2045       infile REFFILE truncover;
2046       input log_line $400.;
2047       I=index((log_line), 'SYSCC=');
2048   run;
2049  
2050   data _null_;
2051       infile REFFILE truncover;
2052       input log_line $400.;
2053       if index((log_line), 'SYSCC=') =1 then call symputx('ERROR_CODE', STRIP(SUBSTR(log_line,8)));
2054   if index((log_line), 'ERROR ') > 0 or index((log_line), 'ERROR:') > 0  then call symputx('ERROR_DESC', STRIP(log_line));
2055   run;
2056  
2057   %if %sysevalf(%superq(ERROR_CODE) > 0) %then %do;
2058       %let Job_status = Failed;
2059   %end;
2060  
2061   %put &ERROR_CODE.;
2062   %put &ERROR_DESC.;
2063   %put &Job_status.;
2064  
2065   proc sql;
2066       update LIB.AUDIT_TBL SET STATUS="&Job_status."
2067       where JOB_NAME = "&jobnm.";
2068       update LIB.AUDIT_TBL SET EXIT_CODE=&ERROR_CODE.
2069       where JOB_NAME = "&jobnm.";
2070   update LIB.AUDIT_TBL SET START_DTTM="&logdt."dt
2071       where JOB_NAME = "&jobnm.";
2072   update LIB.AUDIT_TBL SET END_DTTM="&endt."dt
2073       where JOB_NAME = "&jobnm.";
2074   quit;
2075   %mend m_upd_job_status;
2076  
2077   %m_upd_job_status(m_full_name=&full_job. ,m_job_name=&jobnm.,m_startdt=&logdt.,m_end_dt=&endt.);
MLOGIC(M_UPD_JOB_STATUS):  Beginning execution.
SYMBOLGEN:  Macro variable FULL_JOB resolves to 027_JOB1_02MAY2025:19:18:54
SYMBOLGEN:  Macro variable JOBNM resolves to 027_JOB1
SYMBOLGEN:  Macro variable LOGDT resolves to 02MAY2025:19:18:54
SYMBOLGEN:  Macro variable ENDT resolves to 02MAY2025:19:18:55
MLOGIC(M_UPD_JOB_STATUS):  Parameter M_FULL_NAME has value 027_JOB1_02MAY2025:19:18:54
MLOGIC(M_UPD_JOB_STATUS):  Parameter M_JOB_NAME has value 027_JOB1
MLOGIC(M_UPD_JOB_STATUS):  Parameter M_STARTDT has value 02MAY2025:19:18:54
MLOGIC(M_UPD_JOB_STATUS):  Parameter M_END_DT has value 02MAY2025:19:18:55
SYMBOLGEN:  Macro variable M_FULL_NAME resolves to 027_JOB1_02MAY2025:19:18:54
MPRINT(M_UPD_JOB_STATUS):   FILENAME REFFILE FILESRVC FOLDERPATH='/Project/Test/LOGS'
FILENAME="027_JOB1_02MAY2025:19:18:54.txt";
MLOGIC(M_UPD_JOB_STATUS):  %LET (variable name is JOB_STATUS)
MLOGIC(M_UPD_JOB_STATUS):  %LET (variable name is ERROR_CODE)
MLOGIC(M_UPD_JOB_STATUS):  %LET (variable name is ERROR_DESC)
MPRINT(M_UPD_JOB_STATUS):   data test;
MPRINT(M_UPD_JOB_STATUS):   infile REFFILE truncover;
MPRINT(M_UPD_JOB_STATUS):   input log_line $400.;
MPRINT(M_UPD_JOB_STATUS):   I=index((log_line), 'SYSCC=');
MPRINT(M_UPD_JOB_STATUS):   run;
NOTE: The data set WORK.TEST has 0 observations and 2 variables.
WARNING: Data set WORK.TEST was not replaced because this step was stopped.
NOTE: DATA statement used (Total process time):
      real time           0.00 seconds
      cpu time            0.01 seconds
     
MPRINT(M_UPD_JOB_STATUS):   data _null_;
MPRINT(M_UPD_JOB_STATUS):   infile REFFILE truncover;
MPRINT(M_UPD_JOB_STATUS):   input log_line $400.;
MPRINT(M_UPD_JOB_STATUS):   if index((log_line), 'SYSCC=') =1 then call symputx('ERROR_CODE', STRIP(SUBSTR(log_line,8)));
MPRINT(M_UPD_JOB_STATUS):   if index((log_line), 'ERROR ') > 0 or index((log_line), 'ERROR:') > 0 then call symputx('ERROR_DESC',
STRIP(log_line));
MPRINT(M_UPD_JOB_STATUS):   run;
NOTE: DATA statement used (Total process time):
      real time           0.00 seconds
      cpu time            0.00 seconds
     
MLOGIC(M_UPD_JOB_STATUS):  %IF condition %sysevalf(%superq(ERROR_CODE) > 0) is FALSE
MLOGIC(M_UPD_JOB_STATUS):  %PUT &ERROR_CODE.
SYMBOLGEN:  Macro variable ERROR_CODE resolves to
MLOGIC(M_UPD_JOB_STATUS):  %PUT &ERROR_DESC.
SYMBOLGEN:  Macro variable ERROR_DESC resolves to
MLOGIC(M_UPD_JOB_STATUS):  %PUT &Job_status.
SYMBOLGEN:  Macro variable JOB_STATUS resolves to Completed
Completed
MPRINT(M_UPD_JOB_STATUS):   proc sql;
NOTE: PROC SQL set option NOEXEC and will continue to check the syntax of statements.
SYMBOLGEN:  Macro variable JOB_STATUS resolves to Completed
SYMBOLGEN:  Macro variable JOBNM resolves to 027_JOB1
MPRINT(M_UPD_JOB_STATUS):   update LIB.AUDIT_TBL SET STATUS="Completed" where JOB_NAME = "027_JOB1";
 
ORACLE_379: Prepared: on connection 4
SELECT * FROM LIB.AUDIT_TBL
 
 
ORACLE_380: Prepared: on connection 4
update LIB.AUDIT_TBL set LIB.AUDIT_TBL."STATUS" = 'Completed' where LIB.AUDIT_TBL."JOB_NAME" =
'027_JOB1'
 
NOTE: Statement not executed due to NOEXEC option.
SYMBOLGEN:  Macro variable ERROR_CODE resolves to
SYMBOLGEN:  Macro variable JOBNM resolves to 027_JOB1
NOTE 137-205: Line generated by the invoked macro "M_UPD_JOB_STATUS".
2077     proc sql;     update LIB.AUDIT_TBL SET STATUS="&Job_status."     where JOB_NAME = "&jobnm.";     update
2077 ! LIB.AUDIT_TBL SET EXIT_CODE=&ERROR_CODE.     where JOB_NAME = "&jobnm.";  update LIB.AUDIT_TBL SET
                                                                     --------
                                                                     22
ERROR 22-322: Syntax error, expecting one of the following: (, ',', WHERE.  
MPRINT(M_UPD_JOB_STATUS):   update LIB.AUDIT_TBL SET EXIT_CODE= where JOB_NAME = "027_JOB1" ;
SYMBOLGEN:  Macro variable LOGDT resolves to 02MAY2025:19:18:54
SYMBOLGEN:  Macro variable JOBNM resolves to 027_JOB1
MPRINT(M_UPD_JOB_STATUS):   update LIB.AUDIT_TBL SET START_DTTM="02MAY2025:19:18:54"dt where JOB_NAME =
"027_JOB1";
 
ORACLE_381: Prepared: on connection 4
SELECT * FROM LIB.AUDIT_TBL
 
 
ORACLE_382: Prepared: on connection 4
update LIB.AUDIT_TBL set LIB.AUDIT_TBL."START_DTTM" = TO_DATE('  
02MAY2025:19:18:54','DDMONYYYY:HH24:MI:SS','NLS_DATE_LANGUAGE=American') where LIB.AUDIT_TBL."JOB_NAME" =
'027_JOB1'
 
NOTE: Statement not executed due to NOEXEC option.
SYMBOLGEN:  Macro variable ENDT resolves to 02MAY2025:19:18:55
SYMBOLGEN:  Macro variable JOBNM resolves to 027_JOB1
MPRINT(M_UPD_JOB_STATUS):   update LIB.AUDIT_TBL SET END_DTTM="02MAY2025:19:18:55"dt where JOB_NAME =
"027_JOB1";
 
ORACLE_383: Prepared: on connection 4
SELECT * FROM LIB.AUDIT_TBL
 
 
ORACLE_384: Prepared: on connection 4
update LIB.AUDIT_TBL set LIB.AUDIT_TBL."END_DTTM" = TO_DATE('  
02MAY2025:19:18:55','DDMONYYYY:HH24:MI:SS','NLS_DATE_LANGUAGE=American') where LIB.AUDIT_TBL."JOB_NAME" =
'027_JOB1'
 
NOTE: Statement not executed due to NOEXEC option.
MPRINT(M_UPD_JOB_STATUS):   quit;
NOTE: The SAS System stopped processing this step because of errors.
NOTE: PROCEDURE SQL used (Total process time):
      real time           0.01 seconds
      cpu time            0.00 seconds
     
MLOGIC(M_UPD_JOB_STATUS):  Ending execution.

[ballardw]

Your Work.test has zero observations. And the note that the step was halted is an indication that perhaps the FILENAME didn't execute properly. Is Project in that path the root or mount point of a drive? If not, the folderpath may have been treated as relative to where the program executed and the file wasn't found.

MPRINT(M_UPD_JOB_STATUS):   data test;
MPRINT(M_UPD_JOB_STATUS):   infile REFFILE truncover;
MPRINT(M_UPD_JOB_STATUS):   input log_line $400.;
MPRINT(M_UPD_JOB_STATUS):   I=index((log_line), 'SYSCC=');
MPRINT(M_UPD_JOB_STATUS):   run;
NOTE: The data set WORK.TEST has 0 observations and 2 variables.
WARNING: Data set WORK.TEST was not replaced because this step was stopped.
NOTE: DATA statement used (Total process time):
      real time           0.00 seconds
      cpu time            0.01 seconds

 Additionally the "not replaced" might mean that an early version of the Work.Test was used in later steps.

[Tom]

Probably because the error caused SAS to set syntaxcheck mode.

Note this hint in your LOG.

MPRINT(M_UPD_JOB_STATUS):   proc sql;
NOTE: PROC SQL set option NOEXEC and will continue to check the syntax of statements.

That might also explain why you read 0 observations from the text file without it saying the file could not be found.

You might try changing setting the options NOSYNTAXCHECK before the step that is causing the errors.  https://documentation.sas.com/doc/en/pgmsascdc/9.4_3.5/lesysoptsref/n014qbvh3po8w5n1qlqbzr22vtg0.htm

But in general it is better to run the code that checks the logs generated by your SAS program from a separate program that runs after to SAS program finishes to avoid that type of issue.

[Quentin · Accepted solution]

Agree with Tom's diagnosis that the likely cause is an error in the program the pushed the session into syntaxcheck mode, which will not execute steps beyond that point.  In syntaxcheck mode, SAS sets the system options obs=0 and NOREPLACE.

If you want to run a log check at the end of the program, you can recover from syntaxcheck mode by setting options:

 options obs=max replace NoSyntaxCheck;

You could do that at the beginning of your macro.