DATA Step, Macro, Functions and more

trying to concatenate 400 files using macro

Accepted Solution Solved
Reply
Occasional Contributor SN
Occasional Contributor
Posts: 7
Accepted Solution

trying to concatenate 400 files using macro

I have 400 files to merge as one dataset. They have the same columns, but are of different subjects. So every line in each of the data set needs to be included.

not sure if the "merge by" would work here. Please help


Accepted Solutions
Solution
‎03-05-2013 10:41 AM
Trusted Advisor
Posts: 1,614

Re: trying to concatenate 400 files using macro

Create a macro variable &names that contains all the data set names, replace yourlibnamegoeshere with the actual name of the library, all in UPPER CASE

proc sql noprint;

     select distinct memname into :names separated by ' ' from sashelp.vtable where libname="yourlibnamegoeshere";

quit;

Then concatenate all data sets

data all;

     set &names;

...

View solution in original post


All Replies
Respected Advisor
Posts: 4,646

Re: trying to concatenate 400 files using macro

You might not need macro at all.  If your files are text you could read them all in one datastep using wildcards in the infile statement (at least on Windows).

PG

PG
Occasional Contributor SN
Occasional Contributor
Posts: 7

Re: trying to concatenate 400 files using macro

they are as SAS datasets in a lib ; Number of rows varies from 1  to 50  rows  in the datasets. Finally, after combining the datasets, I would need to run stats analysis on it.

Trusted Advisor
Posts: 1,614

Re: trying to concatenate 400 files using macro

I said "depending on how the files are named"

You didn't address this. Are they named sequentially, or by some identifier, or otherwise? What are some of the names?

Trusted Advisor
Posts: 1,614

Re: trying to concatenate 400 files using macro

It's not clear that "MERGE" is the way to go. I think SET makes more sense.

Are these 400 files SAS data sets, or external files?

Depending on how the files are named, and what type of files you are working with, we can probably provide more specific advice.

Occasional Contributor SN
Occasional Contributor
Posts: 7

Re: trying to concatenate 400 files using macro

Each dataset is a subject data labelled alphanumerically in a random order. There is no pattern in the dataset names.

I have a list of the 400 names in a seperate dataset. I need to write a script which would do this

data=combineddata;

set ds1...ds400;

How do I use macro to call each of the file from the filename dataset and combine them together.

Solution
‎03-05-2013 10:41 AM
Trusted Advisor
Posts: 1,614

Re: trying to concatenate 400 files using macro

Create a macro variable &names that contains all the data set names, replace yourlibnamegoeshere with the actual name of the library, all in UPPER CASE

proc sql noprint;

     select distinct memname into :names separated by ' ' from sashelp.vtable where libname="yourlibnamegoeshere";

quit;

Then concatenate all data sets

data all;

     set &names;

...

Occasional Contributor SN
Occasional Contributor
Posts: 7

Re: trying to concatenate 400 files using macro

Thanks much. this should work.

But I am currently getting an error for most of the variables

ERROR: Variable SBP has been defined as both character and numeric.

ERROR: Variable DBP has been defined as both character and numeric.

Probably the  way forward is to check all the files for the variable type and run this code again.

r

Occasional Contributor SN
Occasional Contributor
Posts: 7

Re: trying to concatenate 400 files using macro

Thanks much. The code worked. I found a couple of empty files which gave an error as above.

Respected Advisor
Posts: 3,777

Re: trying to concatenate 400 files using macro

With 400 datasets on the SET statement you may want to look at the SET statement documentation concerning the OPEN=DEFER option.

☑ This topic is SOLVED.

Need further help from the community? Please ask a new question.

Discussion stats
  • 9 replies
  • 282 views
  • 3 likes
  • 4 in conversation