DATA Step, Macro, Functions and more

Append several csv files with many variables

Accepted Solution Solved
Reply
Frequent Learner
Posts: 1
Accepted Solution

Append several csv files with many variables

Hello,

 

My problem seems simple but I am quite new to SAS so I would appreciate some help.

I need to create a dataset from several CSV files that appear in a directory. It seems that it is easy to use a command data-infile-input to do that. However, the problem is that I have around 90 variables (columns) so writing each one of those next to the input statement seems quite inefficient.

 

Is there any other way to append all these csv files and keep all the variables available?

 

Thanks


Accepted Solutions
Solution
‎07-06-2016 03:03 PM
Contributor
Posts: 25

Re: Append several csv files with many variables

Although writing out the full INPUT statement may seem 'inefficient', it is really best practice because it gives you full control. Also, you may be able to generate the INPUT statement if you have the field names available in electronic form.

 

The lazy, er, efficient approach will be to use PROC IMPORT as 'CSV' is one of the available DBMS options.

 

Hope this sends you on the right solution path.

View solution in original post


All Replies
Solution
‎07-06-2016 03:03 PM
Contributor
Posts: 25

Re: Append several csv files with many variables

Although writing out the full INPUT statement may seem 'inefficient', it is really best practice because it gives you full control. Also, you may be able to generate the INPUT statement if you have the field names available in electronic form.

 

The lazy, er, efficient approach will be to use PROC IMPORT as 'CSV' is one of the available DBMS options.

 

Hope this sends you on the right solution path.

Super User
Super User
Posts: 7,083

Re: Append several csv files with many variables

If you have metadata on the files (note that CSV files have no place to store metadata, at best the CSV might have column names in the first row) then you can easily use it to generate the code to read the files.

 

If you have no information on what is in the files you can use PROC IMPORT and SAS will make an intellegent guess as to what type of fields you have basd on the data in the file and generate a data step to read the file.

 

If the files are all in the same format you can read them all in a single data step if you want.  If you don't care what the variable names are you can just name them something like VAR1 to VAR90.

 

data want ;
   infile 'my_directory/*.csv' dsd firstobs=2 truncover eov=eov;
   input @;
   if eov then input;
   eov=0;

   length var1-var90 $100 ;
   input var1-var90 ;
run;

 

☑ This topic is solved.

Need further help from the community? Please ask a new question.

Discussion stats
  • 2 replies
  • 247 views
  • 1 like
  • 3 in conversation