DATA Step, Macro, Functions and more

arrays efficiency problem

Accepted Solution Solved
Reply
New Contributor
Posts: 4
Accepted Solution

arrays efficiency problem

the following code works fine. but when I increase the &col = 5000, it becomes very redundant. Any suggestions that I can do something like symput so save some space?


%let row = 10;

%let col = 5;

data test1;

     do i = 1 to &row.;

          output;

     end;

run;

data test2;

     set test1;

     array econ econ_1 - econ_&col.;

     array z z_1 - z_&col.;

     do j = 1 to &col.;

          if _n_ = 1 then econ{j} = rannor(1);

          z{j} = econ{j} * i;

     end;

     retain econ_1 - econ_&col.;

run;


Accepted Solutions
Solution
‎09-14-2012 03:32 PM
New Contributor
Posts: 4

Re: arrays efficiency problem

Got help from SAS already. See below code. Appreciate your time.

%let col = 3;

Data _null_;

     do i = 1 to &col.;

          call symput ('econ_'||strip(i), rannor(1));

     end;

run;

%macro test;

data test2;

     set test1;

     array z z_1 - z_&col.;

     %do j = 1 %to &col.;

          z{&j} = &&econ_&j. *i;

     %end;

run;

%mend;

%test;

View solution in original post


All Replies
Super User
Posts: 11,343

Re: arrays efficiency problem

One minor point if you only have one input set then _N_ will only be one once so there is no reaso to include that test in the DO J= loop.

It sounds like you have a varying number of econ variables and you are sometimes creating extra variables that you don't want.

A quick and dirty approach would be to set COL using the data assuming you have a dataset with more meaningful ECON variables.

data _null_;

     set have (obs=1);

     array econ econ_: ;

     x= dim(econ);

     call symput('col',put(x,f5.0));

run;


New Contributor
Posts: 4

Re: arrays efficiency problem

"One minor point if you only have one input set then _N_ will only be one once so there is no reaso to include that test in the DO J= loop."

Not quite understand what you mean here. The input data test1 has 10 observations. And, _N_ will be used 5 times as there are 5 arrays being created.

"It sounds like you have a varying number of econ variables and you are sometimes creating extra variables that you don't want."

I do want to use those retained observations, but since they are all the same within same econ_*, it is redundent and it takes too much space. So I am thinking if I can write these 5 (&col.) econ_* as global variables, and retreat them when I need them.

Solution
‎09-14-2012 03:32 PM
New Contributor
Posts: 4

Re: arrays efficiency problem

Got help from SAS already. See below code. Appreciate your time.

%let col = 3;

Data _null_;

     do i = 1 to &col.;

          call symput ('econ_'||strip(i), rannor(1));

     end;

run;

%macro test;

data test2;

     set test1;

     array z z_1 - z_&col.;

     %do j = 1 %to &col.;

          z{&j} = &&econ_&j. *i;

     %end;

run;

%mend;

%test;

Super User
Posts: 11,343

Re: arrays efficiency problem

I thought your data was to simulate something in lieue of your actual data.

I do not know what you mean by redundant or what space you want to save.

If you do not want variables in the output data set but need them for calculations during the execution of the data set then drop them from the output.

data test2 (drop= econ_: ) ;

may be what your are looking for.

New Contributor
Posts: 4

Re: arrays efficiency problem

The real job envolved 1 million observation and 10 arrays, each array has a size of 5000. So by elimiating one array (econ, which is a random economic variable shared by all observations) helps to reduce the size dramatically. 'SAS help' provided me a solution (Please see 3. Re.) which is exactly what I mean to do.

🔒 This topic is solved and locked.

Need further help from the community? Please ask a new question.

Discussion stats
  • 5 replies
  • 250 views
  • 0 likes
  • 2 in conversation