topic Re: Reading data from multiple files in SAS Procedures

Reading data from multiple files

BchBnz — Wed, 12 Mar 2014 14:58:18 GMT

Hello everyone,

I hope you are all well.

I have an issue with reading multiple data files in SAS. I tried many methods but it is not working.

Here is an example of the code that I used. It worked with .txt files but I tried only with 5 files.

Method 1:

%let dirname = E:\Data\Lisbon;

filename DIRLIST pipe "dir /B &dirname\*.tas";

data dirlist ;

length fname $256;

infile dirlist length=reclen ;

input fname $varying256. reclen ;

run;

data all_tas_lisbon (drop=fname);

length myfilename $100;

length name $25;

set dirlist;

filepath = "&dirname\"||fname;

infile dummy filevar = filepath length=reclen end=done DSD dlm= ',' TRUNCOVER;

do while(not done);

myfilename = filepath;

length date 8 time 8 cpf $1 cnt $3 ba $1;

informat date ED yymmdd8. time time8.;

format date ED yymmdd10. time time8.;

retain date;

if _n_=1 then do;

input //date//;

end;

input time cpf $ cnt $ ED vol ba $ price var1;

output;

end;

run;

The data was incomplete and there is what I read on the log window:

NOTE: Variable name is uninitialized.

NOTE: The infile DUMMY is:

Filename=E:\Data\Lisbon\s_equities_20080101.tas,

RECFM=V,LRECL=256,File Size (bytes)=60,

Last Modified=04 March 2008 11:10:46 o'clock,

Create Time=11 March 2014 22:27:05 o'clock

NOTE: LOST CARD.

Thank you in advance for your precious time and help.

Re: Reading data from multiple files

Tom — Wed, 12 Mar 2014 15:13:26 GMT

The IF _N_=1 will not work in this case as that will apply only to the first filename they you read from the dataset DIRLIST.

You can fix that by changing the DO loop. Also note the RETAIN is not needed as you are reading the whole file in one iteration of the data step.

myfilename = filepath;

length date 8 time 8 cpf $1 cnt $3 ba $1;

informat date ED yymmdd8. time time8.;

format date ED yymmdd10. time time8.;

do _n_=1 by 1 while (not done);

if _n_=1 then input //date//;

input time cpf $ cnt $ ED vol ba $ price var1 ;

output;

end;

Re: Reading data from multiple files

Astounding — Wed, 12 Mar 2014 15:30:49 GMT

It looks like the program creates FNAME, not NAME. Did you expect NAME to contain any information?

Tom's approach looks good, although I would simplify the DO loop to look like this:

input // date //;

do until (done);

input time cpf $ cnt $ ED vol ba $ price var1;

output;

end;

Good luck.

Re: Reading data from multiple files

BchBnz — Wed, 12 Mar 2014 15:40:42 GMT

Hello Tom,

Thank you for your quick answer.

I modified that section and it work perfectly!

Thank you

Re: Reading data from multiple files

BchBnz — Wed, 12 Mar 2014 15:42:57 GMT

Thank you Astounding,

The NAME variable is just a mistake! My bad!

I tried your code as well and it work just fine.

Many thanks.

Re: Reading data from multiple files

data_null__ — Wed, 12 Mar 2014 15:57:44 GMT

I think @tom use WHILE to prevent lost card message in case the file is empty.