topic Re: How do i read Multiple log files from a path having incremental version number to a dataset in SAS Programming

How do i read Multiple log files from a path having incremental version number to a dataset

Ramag — Fri, 18 Aug 2023 13:21:00 GMT

I Am trying to read all the log files of a user to a dataset having incremental number version .

Please check below example.

I am SAS 9.4 , trying to extract all the log files of user1 having 17 versions into dataset chkmain.

I tried below code, the iterations are working but not getting observations into dataset. Pease suggest.

%macro version(file=,ver=);
data chk_lst;

%do i= 1 %to &ver ;

infile "&pathname&file&i"
    lrecl=900 termstr=CRLF delimiter=' ' truncover;
    input f_1$1-900;
    output;

%end;
    run;

%mend;
%version(file=user1.log@@/main/,ver=17);

Re: How do i read Multiple log files from a path having incremental version number to a dataset

Quentin — Fri, 18 Aug 2023 14:20:24 GMT

Usual advice is to try getting the code working first without macros. It will be easier to debug. Then after it's working without macros (reading says 3 log files), then go back to working on a macro solution.

You can make a single fileref that points to multiple files, and use a single INFILE statement in your DATA step to read data from all the files, like:

filename mylogs ("Q:\junk\log1.log" "Q:\junk\log2.log") ;
data _null_ ;
  infile mylogs ;
  input ;
  put _infile_ ;
run ;

If you can get a step like that working, then your macro is simpler to write, because it only needs to generate the list of file names.

Re: How do i read Multiple log files from a path having incremental version number to a dataset

Patrick — Sat, 19 Aug 2023 02:08:39 GMT

Did you consider using a wildcard?

%let log_path=c:\temp;
%let log_rootname=mylog;

data work.want;
  length _fname $400 log_name $80 line_num 8;
  infile "&log_path\&log_rootname.*.log" truncover filename=_fname;
  input logline $char256.;
  log_name=scan(_fname,-1,'\/');
  if lag(log_name) ne log_name then line_num=1; else line_num+1;
run;

Re: How do i read Multiple log files from a path having incremental version number to a dataset

Kurt_Bremser — Sat, 19 Aug 2023 07:17:45 GMT

What does the log say?

Re: How do i read Multiple log files from a path having incremental version number to a dataset

Kurt_Bremser — Sat, 19 Aug 2023 07:27:11 GMT

You use slashes for the path, indicating a UNIX file system. UNIX uses LF as TERMSTR.

Since you create this code

data ...;
infile ...;
input ...;
output;
infile ...;
input ...;
output;
/* and so on */
run;

any input trying to read past the end of any one file will terminate the step, even if the other files still have data.

To read multiple external files in a single DATA step, you must use only one INFILE statement which opens all the files in succession, either by supplying a list of names and using the FILEVAR= option, or by using wildcards.

Re: How do i read Multiple log files from a path having incremental version number to a dataset

Tom — Sat, 19 Aug 2023 20:01:59 GMT

The macro is not generating the right code.

But you don't need macro logic to read 17 different files in a data step. Just data step logic.

%let pathname=some direcotry/;
%let file=some prefix;
%let ver=17;


data chk_lst;
  length ver 8 fname $50 filename $256 lineno 8 line $900;
  do ver=1 to &ver ;
    fname=cats("&file",ver);
    filename=cats("&pathname",fname);
    infile logfile filevar=filename end=eof truncover;
    do lineno=1 by 1 while (not eof);
      input line $char900.;
      output;
    end;
  end;
run;