topic splitting data in SAS Programming

splitting data

hhchenfx — Thu, 23 Feb 2017 17:01:45 GMT

Hi,

I have a big file and want to split it into different smaller file with the same record and fewer column each file.

In the below sample, I will split have into b and c file.

My issue is that I want to delete the columns already used so the have file becomes smaller after each round.

Of course I can do something like " data have; set have; drop..." but it takes time to run that data step.

Thank you.


data have;
input x b1 b11 b2 b22;
datalines;
1 5 10 20 1
2 5 1 2 2
3 50 100 200 3
;run;

%macro split_data (rr=);
data data_&rr ; set have;
keep x b&rr.: ;run; 
%mend;
%split_data (rr=1);
%split_data (rr=2);
run;

Re: splitting data

Reeza — Thu, 23 Feb 2017 17:16:35 GMT

If you're splitting your table via columns you may as well set up a fact/dim table structure using a star or snowflake schema.

If thats an option.

What makes your file 'big'?

Re: splitting data

art297 — Thu, 23 Feb 2017 17:32:01 GMT

Why not do it all in one data step? e.g.,

data have;
input x b1 b11 b2 b22;
datalines;
1 5 10 20 1
2 5 1 2 2
3 50 100 200 3
;
run;

data data1 (keep=x b1--b11)
         data2 (keep=x b2--b22);
  set have;
run;

Art, CEO, AnalystFinder.com

Re: splitting data

hhchenfx — Thu, 23 Feb 2017 22:51:10 GMT

Perfect, Art297.
The combine all in 1 data step save a lot of time.
HHC