DATA Step, Macro, Functions and more

Concatenating 1000 datasets together

Reply
N/A
Posts: 0

Concatenating 1000 datasets together

Hi, I have 1000 datasets with 1 row in each dataset and they are named in a similar fashion like this.
seg_1 seg_2 seg_3..........seg_1000.
I'm trying to concatenate all these 1000 datasets in one data step.

%MACRO MERGING;
DATA ALL_SEG;
%DO q = 1 %TO 1000;
SET seg_&q. ;
%END;
RUN;
%MEND;
%MERGING;

But I'm getting only 1 row in my final dataset (all_seg).
Can somebody pls help me wiht this.( I dont want to write all the 1000 datset names after the SET Statement).

Thanks
Respected Advisor
Posts: 3,799

Re: Concatenating 1000 datasets together

Posted in reply to deleted_user
with SAS9.2 you can use : in set statement set seq_:; but I don't have 9.2

[pre]
* Make test data;
data _null_;
do _n_ = 1 to 1000;
call execute(catx(' ',cats('data seq_',_n_),'; retain x',_n_,'; run;'));
end;
run;

* hook em up;
data _null_;
call execute('data SEQ0; set ');
do _n_ = 1 to 1000;
call execute(cats('seq_',_n_));
end;
call execute(' open=defer; run;');
run;
[/pre]
Frequent Contributor
Frequent Contributor
Posts: 77

Re: Concatenating 1000 datasets together

Posted in reply to deleted_user
Try moving the set and the semi-colon outside of the %do loop

%MACRO MERGING;
DATA ALL_SEG;
set %DO q = 1 %TO 1000;
seg_&q.
%END; ;
RUN;
%MEND;
%MERGING;
Frequent Contributor
Posts: 81

Re: Concatenating 1000 datasets together

You may also try this but it's not in one data step though...

data _null_;
length all $20000;
do i = 1 to 1000;
all = compbl(all) ||'seg_'||compress(put((i),4.))||' ';
end;
call symput('dsname', all);
run;

%put &dsname;

data final;
set &dsname;
run;
SAS Super FREQ
Posts: 8,864

Re: Concatenating 1000 datasets together

hi:
Consider the following sample data and programs (without involving any SAS macro into the mix)

Which result do you hope to produce? The results from dataset TESTIT or the results from dataset NEWTEST?????

Remember that the SAS Macro facility is not EXECUTING any code for you. It is just doing the typing so your macro program is typing the program code that you want to have sent to the compiler -- when the macro program has finished typing for you.

What is the correct SET statement for what you want to achieve?

cynthia

[pre]
data seg_1;
x=111;
run;

data seg_2;
x=222;
run;

data seg_3;
x=333;
run;

data testit;
set seg_1;
set seg_2;
set seg_3;
run;

proc print data=testit;
title '3 set statements';
run;

data newtest;
set seg_1 seg_2 seg_3;
run;

proc print data=newtest;
title '1 set statement';
run;
[/pre]
N/A
Posts: 0

Re: Concatenating 1000 datasets together

Posted in reply to Cynthia_sas
Hi Cynthia,
I want the results in NEWTEST dataset.

data newtest; set seg_1 seg_2 seg_3;run;

Thanks for your reply..
SAS Super FREQ
Posts: 8,864

Re: Concatenating 1000 datasets together

Posted in reply to deleted_user
Hi:
If that's what you want, then your macro program is incorrect. You need the SET statement (the beginning of the statement) to be OUTSIDE the %DO loop and, you need the ending semicolon for the SET statement to be outside the DO loop.

So, that's the problem with your macro program.

However, the SET statement forces SAS to read each dataset, row, by row -- for 1000 datasets, depending on how big they are, this could take significant time to process. The PROC APPEND method may prove to be for efficient for processing, because the datasets are not read, row by row to construct the new dataset.

You might want to consider the PROC APPEND approach, unless there was other processing you were going to do in the DATA step program.

cynthia
N/A
Posts: 0

Re: Concatenating 1000 datasets together

Posted in reply to Cynthia_sas
Thanks Cynthia.. I have tried the SET statement & it's working..
SAS Employee
Posts: 174

Re: Concatenating 1000 datasets together

Posted in reply to deleted_user
Consider using proc append - not data step.

data seg_1; x=111;run;
data seg_2; x=222;run;
data seg_3; x=333;run;

data all_seg;
attrib x format=8.;
STOP;
run;

%MACRO appending;
%DO q = 1 %TO 3;
proc append data=seg_&q. base=all_seg; run;
%END;
%MEND;

%appending;
N/A
Posts: 0

Re: Concatenating 1000 datasets together

Posted in reply to GertNissen
Thanks GEniz..This one is working too...
Ask a Question
Discussion stats
  • 9 replies
  • 241 views
  • 0 likes
  • 6 in conversation