BookmarkSubscribeRSS Feed
🔒 This topic is solved and locked. Need further help from the community? Please sign in and ask a new question.
nnl3256
Obsidian | Level 7

I have several SAS generated log files in c:\temp i.e. test1.log test2.log... I need a program that reads each log and writes to a SINGLE text file if the log contains patter ERROR:. the code I have can give me a list of log name in log window without format. Is there better way to generate a text file and format better?

%let path=c:\temp;
filename dirlist pipe 'dir "c:\temp" /s';
data dirlist ;
     length logname outname $100;
	 retain cnt;
     infile dirlist length=reclen ;
	 input buffer $varying256. reclen ;
	 
	 if index(buffer, ".log") gt 0 then do;
	 	cnt+1;
		call symput('totallog',cnt);
	 	logname ="&path\"||strip(scan( buffer, -1, ' ' ));
		s1=scan(logname, -1, '\');
  		outname=scan(s1, 1, '.');
		output;
	end;
	keep logname outname;
run ;

%put &totallog;
%let cnt=&totallog;

proc sql noprint; select logname, outname into 	:log1-:log&cnt, :out1-:out&cnt from dirlist; quit;  


%macro read;
%do i=1 %to &totallog;
	data out_&&out&i; 
	 	 infile "&&log&i" lrecl=40 pad;
	 	 input @1 line $; 
		 if  upcase(substr(left(line),1,6))='ERROR:' then output;
	run;

	proc sql noprint; select line from out_&&out&i; quit; 
	%if &sqlobs gt 0 %then %do;
		%let all=&all. out_&&out&i;
	%end;
%end;
%mend;

%let all=;  %put &all;
%read;
%put ***********Error files: &all***********;
1 ACCEPTED SOLUTION

Accepted Solutions
Tom
Super User Tom
Super User

There are options you can add to your DIR command to make that part easier.  To get just the names of files add the /b and /a-d options.

dir /s/b/a-d

There are easier ways to read the filenames. Use the TRUNCOVER option on the INFILE statement.

infile "dir c:\temp /s/b/a-d" pipe truncover ;
input filename $256. ;	 

Also if you are looking for actual SAS error messages in the log then you should look for lines that start with 'ERROR' and also contain and ':'.   Sometimes SAS will insert information between the 'ERROR' and the colon.

 

Making sure it starts in column 1 will eliminate many false positives from lines that are just echoing code that might be used by a programmer to generate an error message like 

if age < 0 then put 'ERROR: Invalid age. ' age=;

So putting it together you want something like this:

%let path=c:\temp;

data logfiles ;
  infile %sysfunc(quote(dir "&path" /b/s/a-d)) pipe truncover ;
  input filen $256. ;
  filename=filen ;
  if index(filename,'.') and lowcase(scan(filename,-1,'.'))='log' ;
  infile saslog filevar=filen end=eof ;
  nerrors=0;
  do lineno=1 by 1 while (not eof);
    input;
    if _infile_=:'ERROR' and index(_infile_,':') then do;
      if not nerrors then put filename= ;
      putlog lineno= _infile_ ;
      nerrors+1;
    end;
  end;
  keep filename nerrors ;
run;

If you just want the filenames with errors then you can make it even simpler and stop reading the LOG file when you get a hit. 

%let path=c:\temp;

data logfiles ;
  infile %sysfunc(quote(dir "&path" /b/s/a-d)) pipe truncover ;
  input filen $256. ;
  filename=filen ;
  if index(filename,'.') and lowcase(scan(filename,-1,'.'))='log' ;
  infile saslog filevar=filen end=eof ;
  anyerrors=0;
  do while (not eof and not anyerrors);
    input;
    if _infile_=:'ERROR' and index(_infile_,':') then anyerrors=1;
  end;
  if anyerrors;
  keep filename ;
run;

 

View solution in original post

5 REPLIES 5
ErikLund_Jensen
Rhodochrosite | Level 12

Hi

 

I will suggest a different approach with a loop over a data set with file names. I have made an example based on code I am using. For each log file, the erroneus records are appended to the output file using the MOD file option. The code will not find all errors, because some sas errors are reported in the format ERROR 3-20: or some other number. So using prxmatch with a suitable pattern will find all errors. I will be happy to ansver further questions.

%let path=e:\logs\batchjobs;
%let outfile = e:\work\errorlist.txt;
filename dirlist pipe "dir ""&path"" /s";

%let filesfound = 0;
data dirlist (drop=rec folder file filesfound);  	
	length fullfilename folder file $255;
	retain folder;
	infile dirlist truncover end=eof;
	input rec $char300.;
	if left(rec) =: 'Directory of ' then folder = substr(rec,15);
	else if substr(rec,1,1) ne '' then do;
		file = substr(rec,37);
			if file not in ('.','..') then do;
			fullfilename = trim(folder)||'\'||substr(rec,37);
			filesfound + 1;
			output;
		end;
	end;
	if eof then call symputx('filesfound',filesfound);
run;
%put &=filesfound;

%macro read;
	%if &filesfound > 0 %then %do;

		* Get max filename length for formatting output;
		proc sql noprint;
			select max(length(fullfilename))  into :maxlen from dirlist;
		quit;
		%put &=maxlen;

		* Initiate output file;
		data _null_;
			file "&outfile";
			datetime = datetime();
			put "* Error file list generated at " datetime datetime.;
			put "* Error File" @%eval(&maxlen+2) 'Recno' @+2 'Error Line';
		run;

		* loop over files;
		%do i=1 %to &filesfound;

			data _null_; set dirlist (firstobs=&i obs=&i);
				call symput('thisinfile',fullfilename);
			run;

			data _null_;
				 infile "&thisinfile" lrecl=255;
				 file "&outfile" mod;
			 	 input;
				 recno = _N_;
				 if _infile_ =: 'ERROR:' then put "&thisinfile" @%eval(&maxlen+2) recno 5. @+2 _infile_;
			run;
		%end;
	%end;
%end;
%mend;
%read;
nnl3256
Obsidian | Level 7

Thank you for the quick reply. In my C:\temp directory, there are several log files and no sub-directories. I’ve change path to C:\temp and run your code to generate dataset dirlist.  I got an empty dataset.  

ErikLund_Jensen
Rhodochrosite | Level 12

Hi.

 

Interesting - It Works fine for me.

 

Try to insert a put rec $char200.; statement after the Input statement, It lists all input lines in the log, so you can see the lines that goes into the rest of the code. If you don't get the files listed, then something is wrong with the DIR command, otherwise it should be possible to see where the problem occurs. Use similar put- statements to see whether the if-constructs become true and what you get there. 

Tom
Super User Tom
Super User

There are options you can add to your DIR command to make that part easier.  To get just the names of files add the /b and /a-d options.

dir /s/b/a-d

There are easier ways to read the filenames. Use the TRUNCOVER option on the INFILE statement.

infile "dir c:\temp /s/b/a-d" pipe truncover ;
input filename $256. ;	 

Also if you are looking for actual SAS error messages in the log then you should look for lines that start with 'ERROR' and also contain and ':'.   Sometimes SAS will insert information between the 'ERROR' and the colon.

 

Making sure it starts in column 1 will eliminate many false positives from lines that are just echoing code that might be used by a programmer to generate an error message like 

if age < 0 then put 'ERROR: Invalid age. ' age=;

So putting it together you want something like this:

%let path=c:\temp;

data logfiles ;
  infile %sysfunc(quote(dir "&path" /b/s/a-d)) pipe truncover ;
  input filen $256. ;
  filename=filen ;
  if index(filename,'.') and lowcase(scan(filename,-1,'.'))='log' ;
  infile saslog filevar=filen end=eof ;
  nerrors=0;
  do lineno=1 by 1 while (not eof);
    input;
    if _infile_=:'ERROR' and index(_infile_,':') then do;
      if not nerrors then put filename= ;
      putlog lineno= _infile_ ;
      nerrors+1;
    end;
  end;
  keep filename nerrors ;
run;

If you just want the filenames with errors then you can make it even simpler and stop reading the LOG file when you get a hit. 

%let path=c:\temp;

data logfiles ;
  infile %sysfunc(quote(dir "&path" /b/s/a-d)) pipe truncover ;
  input filen $256. ;
  filename=filen ;
  if index(filename,'.') and lowcase(scan(filename,-1,'.'))='log' ;
  infile saslog filevar=filen end=eof ;
  anyerrors=0;
  do while (not eof and not anyerrors);
    input;
    if _infile_=:'ERROR' and index(_infile_,':') then anyerrors=1;
  end;
  if anyerrors;
  keep filename ;
run;

 

nnl3256
Obsidian | Level 7

 Thank you so much, Tom. it works exactly as I expected. 

sas-innovate-2024.png

Join us for SAS Innovate April 16-19 at the Aria in Las Vegas. Bring the team and save big with our group pricing for a limited time only.

Pre-conference courses and tutorials are filling up fast and are always a sellout. Register today to reserve your seat.

 

Register now!

How to Concatenate Values

Learn how use the CAT functions in SAS to join values from multiple variables into a single value.

Find more tutorials on the SAS Users YouTube channel.

Click image to register for webinarClick image to register for webinar

Classroom Training Available!

Select SAS Training centers are offering in-person courses. View upcoming courses for:

View all other training opportunities.

Discussion stats
  • 5 replies
  • 4829 views
  • 0 likes
  • 3 in conversation