topic Re: LSF generated logs having errors and warnings need to capture in SAS Data Management

LSF generated logs having errors and warnings need to capture

SYN — Mon, 04 Feb 2013 11:57:16 GMT

I would like to explain the background.

Example1 - LSF(Platfrom process manger) flow having 5 jobs. Once the flow started and executed fine. all 5 jobs went fine. All 5 job logs are stored in unix server location.

Example2 - LSF(Platfrom process manger) flow having 5 jobs. Once the flow started and executed. 3 jobs went fine and 1 job got ERROR and 1 job got WARNINGS. All 5 job logs are stored in unix server location.

My requirement

I have to take every log once appear in UNIX server location. I have to open the log file and check the log file.

If any ERROR or WARNINGS in log file. I have to capture this log details and write it in one notepad.

I have to write a macro to happen this. Please any suggetions. Thanks in advance.

Re: LSF generated logs having errors and warnings need to capture

LinusH — Mon, 04 Feb 2013 14:30:29 GMT

Been awhile since I worked with LSF, but I guess that it keep log files of flow and job return codes, right?

A nice application would be to have a stored process, that lets you chose between flows (with their RC), and then job within a flow (with their RC), and then link to the actual log.

Re: LSF generated logs having errors and warnings need to capture

Patrick — Tue, 05 Feb 2013 08:23:07 GMT

LSF has a rich set of commanes. BHIST for example lets you query with what return code jobs finished execution. Here the command reference http://support.sas.com/rnd/scalability/platform/PSS6.1/lsf7.06_command_ref.pdf

the second part would then be to parse the logs which ended with an error or warning. You could do this by either writing a SAS program or with some UNIX scripting.

I would bet that if you search the Internet a bit you will find a script which only needs some amendments.

Re: LSF generated logs having errors and warnings need to capture

AndrévandeRiet — Tue, 12 Mar 2013 14:16:12 GMT

Looks to me you want the log messages of the sas jobs and not the LSF log.

This might just do what you wanted.

It captures all the existing logs of sas jobs submitted by LSF (including lsf flowname) and creates a dataset sasuser.errors with the jobname logdate and error lines

You'll have to change things for unix yourself

Hope this helps

    %let SASplatform = d:\SASPLF\;
    %let OTAPLevel   = lev1;
    %let SERVER      = ETL;
    %let LogDir      = BatchServer\logs\;

/*********************************************************************
   * Get the names of the logfiles
   *********************************************************************/
    filename dirlist pipe "dir &SASplatform.GNL&server.\&otaplevel.\SAS&server.\&logdir.*.log";

    data sasuser.logfiles(keep=lognaam);
         length tekst $200 lognaam $120 process $100;
         infile dirlist truncover;
         input tekst $char200.;
         lognaam = "&SASplatform.SAS&server.\&otaplevel.\SAS&server.\&logdir"||scan(tekst,5,' ');
         if index(lognaam,'.log') then
         output;
    run;

/***************************************************************
   * read the logfiles and extract the errors
   ***************************************************************/

   data sasuser.errors(keep=process logdatum line);
     set sasuser.logfiles;
length line $200;
     format datetime datetime 21.2;
     process = scan(scan(lognaam,1,'200'),8,"\");
process = substr(process,1,length(process)-1);
     logdatum = scan(lognaam,-2,'_');
     infile dummy missover truncover end=end length=len filevar=lognaam;
     retain process;
     do until(end);
        input line $200.;
        line = left(line);
        if index(upcase(line),'ERROR:')= 1 then
           do;
*           starttime = input(scan(line,10,' '),?? time8.);
*           startdate = input(scan(line,2,','),date12.);
*           start = dhms(startdatum,0,0,starttijd);
           output sasuser.errors;
           end;
     end; /* end of do until step */
   run;