BookmarkSubscribeRSS Feed
🔒 This topic is solved and locked. Need further help from the community? Please sign in and ask a new question.
OS2Rules
Obsidian | Level 7

Hi all:

Quick question for a Friday afternoon to warm up your thinking devices...

I have a directory with over 9,000 members in it - they are all small (about 25 lines of text) - and the member name contains the date and time stamp for that data.

Currently, I do a "X command" directory listing, and then read that list back in and process the data with a macro that loops through all the members.  The date and time is added to each of the members read and then the whole thing is rolled up into a single SAS table.

I'm wondering if I change this to a "wild card" read using th INFILE statement, is there a way to still retain the date and time values from the member name?  I seem to remember that the name of the member may be in the meta data somewhere, but I can't find any references.

Also - I'm thinking that reading the entire directory using the wild card approach will be much faster, but does anyone have proof?  (I expect that I will have said proof if there is a way for me to use the wild card input....)

Thanks in advance.

1 ACCEPTED SOLUTION

Accepted Solutions
art297
Opal | Level 21

Here is an example of datanull's suggestion.  Try it, compare it with your method, and let us know which works faster:

%let path=c:\art\test\;

/*create some test data;*/

%macro makem;

  %do i=1 %to 1000;

    data _null_;

      file "&path.f&i..txt";

      do j=1 to 1000;

        put j j;

      end;

    run;

  %end;

%mend makem;

%makem

data want;

  length inname $80;

  infile "&path.f*.txt" filename=inname;

  input x y;

  filenm=substr(inname,13);

run;

View solution in original post

4 REPLIES 4
data_null__
Jade | Level 19

Are you reading the data in all files into a single SAS data set? 

For reading from wild card you will need to understand FILENAME= INFILE statement option.  Also you should learn about INFILE statement output EOV=.

Is should be faster than macro looping as you will need only ONE data step to read all files.

art297
Opal | Level 21

Here is an example of datanull's suggestion.  Try it, compare it with your method, and let us know which works faster:

%let path=c:\art\test\;

/*create some test data;*/

%macro makem;

  %do i=1 %to 1000;

    data _null_;

      file "&path.f&i..txt";

      do j=1 to 1000;

        put j j;

      end;

    run;

  %end;

%mend makem;

%makem

data want;

  length inname $80;

  infile "&path.f*.txt" filename=inname;

  input x y;

  filenm=substr(inname,13);

run;

OS2Rules
Obsidian | Level 7

Art:

Just what I was looking for. 

I now have > 16,000 members in the directory that I have to read.  It takes about 20 minutes to read using a macro loop to read each memner, and about 3 minutes using the wildcard method.  

Wild card method is obviously faster (and easier to code!).

Thanks again.

Ksharp
Super User

But wildcard is not as flexiable as the PIPE + filevar=

sas-innovate-2024.png

Join us for SAS Innovate April 16-19 at the Aria in Las Vegas. Bring the team and save big with our group pricing for a limited time only.

Pre-conference courses and tutorials are filling up fast and are always a sellout. Register today to reserve your seat.

 

Register now!

How to Concatenate Values

Learn how use the CAT functions in SAS to join values from multiple variables into a single value.

Find more tutorials on the SAS Users YouTube channel.

Click image to register for webinarClick image to register for webinar

Classroom Training Available!

Select SAS Training centers are offering in-person courses. View upcoming courses for:

View all other training opportunities.

Discussion stats
  • 4 replies
  • 878 views
  • 0 likes
  • 4 in conversation