Wildcard Infile Statement: use Logfile NOTE information of <Number of records>

[FK1]

Hello Everyone,

I am using a wildcard infile statement to read LSF-Logs into SAS:

filename _Harchiv "/path/to/lsf_history_files/pm/work/history/archiv/history.log.*";

data work.Historyfiles_archiv;
  length      historyfile $100.	 ;
/* store the name of the current infile */  
infile _Harchiv filename=historyfile truncover  dlm='20'x; 
input rawdata $1000.   ;
name_history_file = historyfile;
Einlesezeitpunkt_history_file = datetime();
format Einlesezeitpunkt_history_file datetime25.6;
run;

filename _Harchiv clear;

This works perfectly, as the SAS Logs confirms:

NOTE: The infile _HARCHIV is:
      Filename=/path/to/lsf_history_files/pm/work/history/archiv/history.log.1000,
      File List=/path/to/lsf_history_files/pm/work/history/archiv/history.log.*,
      Owner Name=sas,Group Name=sas,
      Access Permission=-rw-r--r--,
      Last Modified=18. November 2021 20.59 Uhr,
      File Size (bytes)=500147

NOTE: The infile _HARCHIV is:
      Filename=/path/to/lsf_history_files/pm/work/history/archiv/history.log.1001,
      File List=/path/to/lsf_history_files/pm/work/history/archiv/history.log.*,
      Owner Name=sas,Group Name=sas,
      Access Permission=-rw-r--r--,
      Last Modified=18. November 2021 22.26 Uhr,
      File Size (bytes)=500674

NOTE: The infile _HARCHIV is:
      Filename=/path/to/lsf_history_files/pm/work/history/archiv/history.log.1002,
      File List=/path/to/lsf_history_files/pm/work/history/archiv/history.log.*,
      Owner Name=sas,Group Name=sas,
      Access Permission=-rw-r--r--,
      Last Modified=18. November 2021 23.56 Uhr,
      File Size (bytes)=500385

...

Also, there are further notes, about the number of records (i.e. lines of the LSF logfile) that have been read in:

NOTE: 1644 records were read from the infile _HARCHIV.
The minimum record length was 77.
The maximum record length was 948.
NOTE: 1627 records were read from the infile _HARCHIV.
The minimum record length was 81.
The maximum record length was 948.
NOTE: 1631 records were read from the infile _HARCHIV.
The minimum record length was 81.
The maximum record length was 948.

My question is:

As SAS obviously derives the number of lines in the LSF Logfile(s), which have been read in, in order to put it in the SAS Log as a NOTE, is there a way, I am able to generate a variable in the SAS dataset that holds the read in infile information, including a variable that tells me, how many lines have been read in using the above mentioned number?

I know in a "regular" data step I can use "nobs=" option in the set statement, or use the internal Variable "_N_".

Since I am using a wildcard infile statement, is there an infile statement option that "gives" me the number of read in lines? The only other way I know of, would be to use a dynamic pipe...

Any help would be highly appreciated.

FK1

[data_null__]

You can use the INFILE statement option EOV= to detect when the file changes.  With that you can count the number of records read.

[Tom · Accepted solution]

Just check when the variable populated by FILENAME= option changes.  I find that is easier to use than the EOV= option variable.

data work.Historyfiles_archiv;
  length historyfile $100;
  infile "/path/to/lsf_history_files/pm/work/history/archiv/history.log.*"
    filename=historyfile truncover 
  ; 
  input rawdata $1000.;
  name_history_file = historyfile;
  if historyfile ne lag(historyfile) then lineno=0;
  lineno+1;
  Einlesezeitpunkt_history_file = datetime();
  format Einlesezeitpunkt_history_file datetime25.6;
run;

[FK1]

Hi @Tom ,

great, it works fine!

@data_null__ : thank you, too, for enlightening me about "EOV" option!