Requesting ideas to efficiently read multiple external files?

[novinosrin]

Hello All,  my objective is read a set of files(delimited) from a folder. The physical filename of the files have varying patterns and the need for me is to read certain files that follows the suit with the same pattern.

First of all, I am trying find ways of taking advantage of FILENAME statement pointing an aggregate location and read files in one shot of course with FILEVAR=. The plan seems ok but I first want to  take a look at the list of filenames in the directory in programmed style unlike opening the folders manually to check.

1. is there an easy straight forward way to list all the files in the directory?

2. Once I can get 1 to work, I would read that using INPUT, write some REGEX to match the pattern and call that using FILEVAR= and be over with it.

Alternatively is there any better or efficient way to do the same that you could recommend? Please point me to the documentation if any. Thank you!

[Reeza · Accepted solution]

1. http://support.sas.com/kb/45/805.html

https://documentation.sas.com/?docsetId=mcrolref&docsetTarget=n0js70lrkxo6uvn1fl4a5aafnlgt.htm&docse...

2. Limit your list as you read it in to match the pattern and then pass that to the FILEVAR option.

You can do this in one or data steps really. I'd do it in two because it's easier to debug and manage that way.

Using a wildcard is also an option, I think you know this already though

https://communities.sas.com/t5/SAS-Communities-Library/How-do-I-write-a-macro-to-import-multiple-tex...

[novinosrin]

"2. Limit your list as you read it in to match the pattern and then pass that to the FILEVAR option." -- Slick presence of mind! 

I knew of the wild card, but honestly didn't cross my mind. Thank you!

---

@Reeza wrote:

1. http://support.sas.com/kb/45/805.html

https://documentation.sas.com/?docsetId=mcrolref&docsetTarget=n0js70lrkxo6uvn1fl4a5aafnlgt.htm&docse...

2. Limit your list as you read it in to match the pattern and then pass that to the FILEVAR option.

You can do this in one or data steps really. I'd do it in two because it's easier to debug and manage that way.

Using a wildcard is also an option, I think you know this already though

https://communities.sas.com/t5/SAS-Communities-Library/How-do-I-write-a-macro-to-import-multiple-tex...

---

[tomrvincent]

1. yes, depending on your O/S...which is...?

[novinosrin]

@Reeza   Sorry but a silly question because of my inexperience.  When I open a lot of text files, the header with col names looks like is split inton two lines, but firstobs=2 reads them correctly. Does that mean the values are wrapped?

[Reeza]

Yup. Use a text editor like notepad++ to open text files. Then you can set options such as show invisible characters, such as line feeds and tabs. Makes it easier to debug things.

You can also do this in Notepad by adjusting the word wrap option.

[Ksharp]

data file_name;
rc=filename('fid','c:\temp\');
did=dopen('fid');
do i=1 to dnum(did);
 fname='c:\temp\'||dread(did,i);	output;
end;
run;