Reading All Members in a Directory

[OS2Rules]

Hi all:

Quick question for a Friday afternoon to warm up your thinking devices...

I have a directory with over 9,000 members in it - they are all small (about 25 lines of text) - and the member name contains the date and time stamp for that data.

Currently, I do a "X command" directory listing, and then read that list back in and process the data with a macro that loops through all the members.  The date and time is added to each of the members read and then the whole thing is rolled up into a single SAS table.

I'm wondering if I change this to a "wild card" read using th INFILE statement, is there a way to still retain the date and time values from the member name?  I seem to remember that the name of the member may be in the meta data somewhere, but I can't find any references.

Also - I'm thinking that reading the entire directory using the wild card approach will be much faster, but does anyone have proof?  (I expect that I will have said proof if there is a way for me to use the wild card input....)

Thanks in advance.

[data_null__]

Are you reading the data in all files into a single SAS data set? 

For reading from wild card you will need to understand FILENAME= INFILE statement option.  Also you should learn about INFILE statement output EOV=.

Is should be faster than macro looping as you will need only ONE data step to read all files.

[art297 · Accepted solution]

Here is an example of datanull's suggestion.  Try it, compare it with your method, and let us know which works faster:

%let path=c:\art\test\;

/*create some test data;*/

%macro makem;

  %do i=1 %to 1000;

    data _null_;

      file "&path.f&i..txt";

      do j=1 to 1000;

        put j j;

      end;

    run;

  %end;

%mend makem;

%makem

data want;

  length inname $80;

  infile "&path.f*.txt" filename=inname;

  input x y;

  filenm=substr(inname,13);

run;

[OS2Rules]

Art:

Just what I was looking for. 

I now have > 16,000 members in the directory that I have to read.  It takes about 20 minutes to read using a macro loop to read each memner, and about 3 minutes using the wildcard method.  

Wild card method is obviously faster (and easier to code!).

Thanks again.

[Ksharp]

But wildcard is not as flexiable as the PIPE + filevar=