Reading data from multiple files

[BchBnz]

Hello everyone,

I hope you are all well.

I have an issue with reading multiple data files in SAS. I tried many methods but it is not working. 

Here is an example of the code that I used. It worked with .txt files but I tried only with 5 files.

Method 1:

%let dirname = E:\Data\Lisbon;

filename DIRLIST pipe "dir /B &dirname\*.tas";

data dirlist ;

     length fname $256;

     infile dirlist length=reclen ;

     input fname $varying256. reclen ;

run;

data all_tas_lisbon (drop=fname);

  length myfilename $100;

  length name $25;

  set dirlist;

  filepath = "&dirname\"||fname;

  infile dummy filevar = filepath length=reclen end=done DSD dlm= ',' TRUNCOVER;

  do while(not done);

  myfilename = filepath;

length date 8 time 8 cpf $1 cnt $3 ba $1;

informat date ED yymmdd8. time time8.;

format date ED yymmdd10. time time8.;

retain date;

if _n_=1 then do;

input //date//;

end;

input time cpf $ cnt $ ED vol ba $ price var1;

    output;

  end;

run;

The data was incomplete and there is what I read on the log window:

NOTE: Variable name is uninitialized.

NOTE: The infile DUMMY is:

      Filename=E:\Data\Lisbon\s_equities_20080101.tas,

      RECFM=V,LRECL=256,File Size (bytes)=60,

      Last Modified=04 March 2008 11:10:46 o'clock,

      Create Time=11 March 2014 22:27:05 o'clock

NOTE: LOST CARD.

Thank you in advance for your precious time and help.

[Tom · Accepted solution]

The IF _N_=1 will not work in this case as that will apply only to the first filename they you read from the dataset DIRLIST.

You can fix that by changing the DO loop.  Also note the RETAIN is not needed as you are reading the whole file in one iteration of the data step.

myfilename = filepath;

length date 8 time 8 cpf $1 cnt $3 ba $1;

informat date ED yymmdd8. time time8.;

format date ED yymmdd10. time time8.;

do _n_=1 by 1 while (not done);

   if _n_=1 then input //date//;

   input time cpf $ cnt $ ED vol ba $ price var1 ;

   output;

end;

[BchBnz]

Hello Tom,

Thank you for your quick answer.

I modified that section and it work perfectly!

Thank you

[Astounding]

It looks like the program creates FNAME, not NAME.  Did you expect NAME to contain any information?

Tom's approach looks good, although I would simplify the DO loop to look like this:

input // date //;

do until (done);

   input time cpf $ cnt $ ED vol ba $ price var1;

   output;

end;

Good luck.

[BchBnz]

Thank you Astounding,

The NAME variable is just a mistake! My bad!

I tried your code as well and it work just fine.

Many thanks.

[data_null__]

I think @tom use WHILE to prevent lost card message in case the file is empty.