topic Re: Append several csv files with many variables in SAS Programming

Append several csv files with many variables

aekara — Wed, 06 Jul 2016 18:17:21 GMT

Hello,

My problem seems simple but I am quite new to SAS so I would appreciate some help.

I need to create a dataset from several CSV files that appear in a directory. It seems that it is easy to use a command data-infile-input to do that. However, the problem is that I have around 90 variables (columns) so writing each one of those next to the input statement seems quite inefficient.

Is there any other way to append all these csv files and keep all the variables available?

Thanks

Re: Append several csv files with many variables

sh0e — Wed, 06 Jul 2016 18:23:52 GMT

Although writing out the full INPUT statement may seem 'inefficient', it is really best practice because it gives you full control. Also, you may be able to generate the INPUT statement if you have the field names available in electronic form.

The lazy, er, efficient approach will be to use PROC IMPORT as 'CSV' is one of the available DBMS options.

Hope this sends you on the right solution path.

Re: Append several csv files with many variables

Tom — Wed, 06 Jul 2016 18:41:55 GMT

If you have metadata on the files (note that CSV files have no place to store metadata, at best the CSV might have column names in the first row) then you can easily use it to generate the code to read the files.

If you have no information on what is in the files you can use PROC IMPORT and SAS will make an intellegent guess as to what type of fields you have basd on the data in the file and generate a data step to read the file.

If the files are all in the same format you can read them all in a single data step if you want. If you don't care what the variable names are you can just name them something like VAR1 to VAR90.

data want ;
   infile 'my_directory/*.csv' dsd firstobs=2 truncover eov=eov;
   input @;
   if eov then input;
   eov=0;

   length var1-var90 $100 ;
   input var1-var90 ;
run;