topic Re: I want to read only a few lines from multiple text files in SAS Programming

I want to read only a few lines from multiple text files

Paige — Wed, 01 Dec 2010 13:26:16 GMT

Suppose I have multiple text files in a directory, and I want to read only a few lines from each and then skip the rest of the text file and move on to the next text file.

So, I know I can read multiple files using an INFILE statement something like

INFILE 'c:\data\*.txt';

followed by the proper input statements.

But this reads every line in the file and only goes to the next file when an end-of-file is reached. How can I tell SAS to stop reading one file when a certain text string is found and then advance to the next file without reading each line?

Re: I want to read only a few lines from multiple text files

Peter_C — Wed, 01 Dec 2010 14:57:05 GMT

review the doc on the infile option FILEVAR=

Re: I want to read only a few lines from multiple text files

Ksharp — Thu, 02 Dec 2010 05:36:08 GMT

Hi.
What is your condition of judgement when stop read the rest of text file and continue to read next file?

Re: I want to read only a few lines from multiple text files

sbb — Fri, 03 Dec 2010 02:53:58 GMT

Consider using the DATA step and the INFILE parameters EOV= to detect the new-file start (after reading the first file). And if you need know the current file-name, use the FILENAME= parameter (after declaring a LENGTH for ).

Here's an example to read all files that start with FRED*.TXT -- the multiple PUTLOG statements demonstrates SAS behavior with the EOV= being set on the first-record instance, except for the _N_=1 condition:

data _null_;
length fileinfo $255;
infile 'c:\temp\fred*.txt' end=eof eov=eov filename=fileinfo;
input @;
putlog '>diag-before>' / _all_;
if eov=1 then do;
recnum=0;
eov=0;
end;
recnum+1;
putlog '>diag-after>' / _all_;
run;

Review the SAS 9.2 INFILE statement parameter documentation for more details.

Scott Barry
SBBWorks, Inc.

Re: I want to read only a few lines from multiple text files

Paige — Fri, 03 Dec 2010 13:52:44 GMT

Thanks, Scott, I received a different reply, but this looks interesting as well, and I will test it out.

Re: I want to read only a few lines from multiple text files

Peter_C — Sat, 04 Dec 2010 09:48:35 GMT

when comparing the effect of * in the filename in the infile statement with use of filevar= you should find they both work but being able to switch to the next file at any stage (for example: after reading 5 lines) the FILEVAR= method can read less and so should run faster.