topic Re: Proc Import multiple CSV files in SAS Enterprise Guide

Proc Import multiple CSV files

RensMeerman — Thu, 09 Feb 2017 11:34:02 GMT

Dear experts,

Me, the newbe, has created a macro to read in some csv files. It works but I think the code can be smarter since now I do this manually for something like 30 files which are all located in the same folder. Here an example for reading in 5 files.

the code in SAS EG:

%macro imp(infile);

PROC IMPORT OUT=SAS_TMP.&infile

datafile = "\\DWCPMCP\Data\01_INPUTDOCS\&infile..csv"

DBMS=csv REPLACE;

GETNAMES=YES;

datarow=6;

run;

%mend imp;

%imp(IL_V450)

%imp(IL_V500)

%imp(IL_V100)

%imp(IL_V350)

%imp(IL_V600)

My is, can I make this code less manually intensive? There are more files coming in. Can I import the whole folder in one time instead of typing them out?

Please advice.

Re: Proc Import multiple CSV files

RW9 — Thu, 09 Feb 2017 11:59:50 GMT

Hi,

Its a good idea to do a search before posting a question. There are literaly hundreds of posts on this topic:

https://communities.sas.com/t5/forums/searchpage/tab/message?q=import+multiple+csv+files

You don't need macro, you can do it in a datastep and use wildcards:

infile "<pathtocsvs>\*.csv";

All you have to remember is proper CSV files have headers, so you need to jump one row per file.

Re: Proc Import multiple CSV files

Reeza — Thu, 09 Feb 2017 14:00:01 GMT

I would recommend the methods here. You may need some modification to start at line 6.

Note that proc import guesses types so as the number of files gets larger it's more likely that you'll get inconsistent data types, ie a VAR is char in one dataset and numeric in another.

Using method above requires you explicitly specify it, once.

Re: Proc Import multiple CSV files

Reeza — Tue, 25 Sep 2018 18:55:18 GMT

And best of all the doc has exactly your example

Example 1: Import All CSV Files That Exist within a Directory

This is an annotated example that explains each step of the code. Sample use:

%drive(c:\temp,csv)

Re: Proc Import multiple CSV files

rogerjdeangelis — Thu, 09 Feb 2017 16:24:04 GMT

* create three csvs;

dm "dexport sashelp.cars'd:\csv\cars.csv' replace";
dm "dexport sashelp.class'd:\csv\class.csv' replace";
dm "dexport sashelp.classfit'd:\csv\classfit.csv' replace";

* create three sas datsets;

%symdel fyl; * just in case it exists;
data _null_;
  do csv="classfit","class","cars";
     fyl=catx(' ',"dm 'dimport",cats('"d:\csv\',csv,'.csv"'),csv,"replace';");
     call execute(fyl);
  end;
run;quit;

NOTE: WORK.CLASSFIT data set was successfully created.
NOTE: The data set WORK.CLASSFIT has 19 observations and 10 variables.

NOTE: WORK.CLASS data set was successfully created.
NOTE: The data set WORK.CLASS has 19 observations and 5 variables.

NOTE: WORK.CARS data set was successfully created.
NOTE: The data set WORK.CARS has 428 observations and 15 variables.

Re: Proc Import multiple CSV files

Uzi — Wed, 21 Mar 2018 16:58:34 GMT

Hi,
I have a question:
What should be changed in the code to select specific files from the folder?

Re: Proc Import multiple CSV files

Reeza — Wed, 21 Mar 2018 17:23:41 GMT

@Uzi That depends on what 'specific files' means.

If they have a naming convention or all the same extension those would be different approaches.

@Uzi wrote:
Hi,
I have a question:
What should be changed in the code to select specific files from the folder?

Re: Proc Import multiple CSV files

Uzi — Thu, 22 Mar 2018 02:42:55 GMT

Hey,

sorry for the delay and the lack of detail.
explanation:
I split the run into several computers to shorten the running time.
The result is multiple .csv files in the local of multiple computers.
I have to group them back into one work file.
I can determine the names according to my convenience.

The problem:
There are several runs in each of the folders and I have to choose each time a specific run.
This means that each time I have to see the file names before importing and then choose who to import and group according to the conditions I set.
Before that I used Excel to consolidate the files and I could use a "flag"\"pointer" for the run I wanted to consolidate.
Hope I've been able to explain myself properly.
Thank you in advance for your help!

Re: Proc Import multiple CSV files

Uzi — Thu, 22 Mar 2018 03:11:44 GMT

Small correction:
The name of the files will apply the year, the quarter, and the group from which it came

Re: Proc Import multiple CSV files

Reeza — Thu, 22 Mar 2018 03:28:17 GMT

I'm not understanding your question. I would suggest starting your own thread and explaining in detail..

Re: Proc Import multiple CSV files

Patrick — Thu, 22 Mar 2018 08:20:05 GMT

@Uzi

Please start a new post with your question.

It's sometimes better to tell us what you have and what you need - and only then tell us what you've already tried. You've got a lot of experienced people in this forum and it's sometimes worth to give the big picture as then you might get solution approaches you haven't even thought about.

Re: Proc Import multiple CSV files

Uzi — Thu, 22 Mar 2018 12:38:11 GMT

Thank you very much Patrick I will.