topic Re: Can you create generic code to loop & combine only the files listed in a text doc? in SAS Programming

Can you create generic code to loop & combine only the files listed in a text doc?

BeckyBell2355 — Tue, 08 Mar 2016 21:51:51 GMT

Background: We're trying to create a sas program (9.4) that combines multiple data .txt files (all same format) from one folder to create one large file. The problem is not all .txt files in the folder should be included and the names differ too much that you can't fix it by specifying something like *ais.txt instead of *.txt. We also want to reduce the number of changes needed in the program to run the program for other years. The program below goes to a folder and lists the directory of all .txt files which are then all combined.

Question: Is there a way to use .txt file that lists by line the specific names of the files you want to be combined like this:

R:\AIS\Final Files\2015\ais1.txt

R:\AIS\Final Files\2015\ais2.txt

R:\AIS\Final Files\2015\wierdname1.txt

and the program calls up that file (ais2015directory.txt) and loops through to combine the files listed?

Or is there a better or easier way to do this where we can still use the same program for different years without making a lot of changes each year?

Does any of this make sense? Thank you!


/*!!!!!Change Lib Directory*/

libname AIS15 'R:\AIS\Final Files\2015'; run;

/*!!! Change Directory*/

%let dirname = R:\AIS\Submissions\2015;

filename DIRLIST pipe "dir /B ""&dirname""\*.txt"; **the 2 sets of double quotes are needed here to allow for spaces in our file name;

data dirlist ;

length fname $256; 

infile dirlist length=reclen ;

input fname $varying256. reclen ;

run;




data AIS_auto (drop=fname);

length field1 $1. field2 $4. field3 $8. ;

set dirlist;

 filepath = "&dirname\"||fname;

infile dummy filevar = filepath length=reclen delimiter=',' end=done DSD MISSOVER lrecl=32767 firstobs=2;;

do while(not done);

myfilename = filepath;

input field1 field2 field3;

output;

end;

run;

Re: Can you create generic code to loop & combine only the files listed in a text doc?

Reeza — Tue, 08 Mar 2016 22:38:24 GMT

I've noticed you're using the FILEVAR option - can you filter the list that gets fed to the input process?

Re: Can you create generic code to loop & combine only the files listed in a text doc?

BeckyBell2355 — Wed, 09 Mar 2016 02:31:18 GMT

I think that's what I thought the seperate file which lists each of the files to use would do.

The desire is to reduce the amount of changes in the coding since the file names and locations are different from year to year (and there's 11-13 different files to combine), so we can still use the same program for old and new years.

Thanks.

Re: Can you create generic code to loop & combine only the files listed in a text doc?

Reeza — Wed, 09 Mar 2016 02:32:51 GMT

It should...I don't quite follow how it's not working for you.

Re: Can you create generic code to loop & combine only the files listed in a text doc?

BeckyBell2355 — Wed, 09 Mar 2016 14:50:07 GMT

Could you please show me example code of what you mean? I'm still learning SAS as I go... Thanks