topic Re: Importing multiple CSV files data to single CSV file in SAS Programming

Importing multiple CSV files data to single CSV file

tysonsri — Wed, 14 Sep 2022 20:06:56 GMT

Hello All, I am using SAS 9.4.

I have a 101 CSV files in the directory. I want to combine all csv files into one single CSV file /one dataset .

Everyday one csv file will add into the same directory and I need to combine all csv files into one single csv file again.

Can you please help me with the sas code

Re: Importing multiple CSV files data to single CSV file

ballardw — Wed, 14 Sep 2022 20:35:41 GMT

Do want a SAS data set or a CSV file?

A data step can read multiple CSV files into a single SAS data set IF the CSV files all have the same structure.

Re: Importing multiple CSV files data to single CSV file

tysonsri — Wed, 14 Sep 2022 20:45:39 GMT

I want a single sas dataset.
All 101 csv files are in same structure.

How do we do that in a sas?
comibe all 101 .csv files to one data set

Re: Importing multiple CSV files data to single CSV file

Reeza — Wed, 14 Sep 2022 20:49:57 GMT

Do you have code to read a single CSV file? As a data step?

When you combine all the data do you need to add any information about which file it came from?

Here's a rough idea on how to read all files into a single SAS data set in one step:

https://communities.sas.com/t5/SAS-Communities-Library/How-do-I-write-a-macro-to-import-multiple-text-files-that-have/ta-p/223627

Re: Importing multiple CSV files data to single CSV file

Tom — Wed, 14 Sep 2022 20:57:24 GMT

If all of the CSV files have the same structure and are either all of the CSV files in that folder or all that follow a simple pattern then just use a single data step with a wildcard, * , in the filename.

So something like this:

data want;
    length fname $256;
    infile "\\my directory\*.csv" dsd truncover filename=fname;
    input @;
    if fname ne lag(fname) then delete;
    length firstvar 8 secondvar $40 lastvar 8;
    informat lastvar date.;
    format lastvar date9.;
    input firstvar -- lastvar;
run;

If the files are really large you might need to work harder by keeping track of which files you already read so you know which ones are new, but if the files are not that big then it is probably just easier to read the all each time.