Hi,
I'm just wondering if anyone could help me enhance my code by adding do loop in my code.
%let logpath = /sas/logs/;
%let logdate = %sysfunc(putn(%eval(%sysfunc(today())-1),yymmdd10.));
%logname=logfile_&logdate;
data work.logfile:
infile "&logpath.&logname..log" lrecl=1000
firstobs=2 truncover;
input
date $ 1-23
message $ 24-400:
run;
The code above is extracting the log file yesterday and I have a code that appends it everyday to keep it in a text file. Can someone help me enhance the code above if for example even if I didn't trigger it for 1week when I run the code the days that aren't cover will run without manually setting the date?
The logic that I think of is using a macro that performs a do loop using the difference of the max date in the text file that I have created and today()-1 as the end date.
Thanks.
Albert
You can extract the startdate into a macro variable with SQL:
proc sql noprint;
select max(date) into :startdate from have;
quit;
Then you can repeat your code in a macro:
%let logpath = /sas/logs/;
%macro repeat_import;
%do i = %eval(&startdate+1) %to %eval(%sysfunc(today())-1);
%let logname=%sysfunc(putn(&i,yymmdd10.));
%logname=logfile_&logdate;
data work.logfile:
infile "&logpath.&logname..log" lrecl=1000
firstobs=2 truncover;
input
date $ 1-23
message $ 24-400
:
run;
proc append data=logfile base=have;
run;
%end;
%mend;
%repeat_import
You can extract the startdate into a macro variable with SQL:
proc sql noprint;
select max(date) into :startdate from have;
quit;
Then you can repeat your code in a macro:
%let logpath = /sas/logs/;
%macro repeat_import;
%do i = %eval(&startdate+1) %to %eval(%sysfunc(today())-1);
%let logname=%sysfunc(putn(&i,yymmdd10.));
%logname=logfile_&logdate;
data work.logfile:
infile "&logpath.&logname..log" lrecl=1000
firstobs=2 truncover;
input
date $ 1-23
message $ 24-400
:
run;
proc append data=logfile base=have;
run;
%end;
%mend;
%repeat_import
So, you just want to append all text files in a directory, then get some output from them. Why not:
data work.logs; length log_name date message $500; infile "&logpath.logfile_*.log" lrecl=1000 indsname=fn; input date $ 1-23 message $ 24-400; log_name=fn; run;
This will read all .log files from the given path into one dataset with three columns - filename, date, and message. You can then process that as you want.
Can someone help me enhance the code above if for example even if I didn't trigger it for 1 week when I run the code the days that aren't cover will run without manually setting the date?
I would approach it a different way. Rather than manually setting a date suffix, which may or may not match your filenames, or otherwise fiddling with what has and hasn't been processed, instead assume that whatever files are in your source directory are files that need to be processed. Then, after successful processing, move them to an archive directory.
First, go here and download the dirlist and parmv macros: https://github.com/scottbass/SAS/tree/master/Macro
Make sure they are in your sasautos path (or %include them into your program).
Then, run this code (modify as needed of course):
%dirlist(dir=/sas/logs)
data files / view=files;
length line $1000;
set dirlist;
infile dummy filevar=fullname end=eof firstobs=2;
do until (eof);
input line;
output;
end;
run;
proc append data=files base=mydata;
run;
I don't have access to Unix, but AFAIK the %dirlist macro should work properly there. Let me know if you encounter any issues.
After successful processing of your files, move the processed files from /sas/logs to /sas/logs/archive (or somewhere out of your incoming files location).
If moving the files is undesired, then sure, get the max date of the last processed file from your text file, and apply a where clause to the dirlist. This would rely on datestamped logs to enforce files "greater than" some reference date. Do you need an end date (today()-1) as an upper bound to the files you process? If today's file is in the directory, is it some incomplete file that you don't want to touch until tomorrow?
For enhanced processing, check the value of &syscc before executing proc append and archiving of the files, and only execute that portion of the code if there are no errors.
I'm not sure this needs macro; even %dirlist is not required, it's just a convenience and has some builtin error checking and path parsing.
Hope this helps...
Are you ready for the spotlight? We're accepting content ideas for SAS Innovate 2025 to be held May 6-9 in Orlando, FL. The call is open until September 25. Read more here about why you should contribute and what is in it for you!
Learn how use the CAT functions in SAS to join values from multiple variables into a single value.
Find more tutorials on the SAS Users YouTube channel.