topic Re: arrays efficiency problem in SAS Programming

arrays efficiency problem

Weihong — Fri, 14 Sep 2012 14:55:45 GMT

the following code works fine. but when I increase the &col = 5000, it becomes very redundant. Any suggestions that I can do something like symput so save some space?

%let row = 10;

%let col = 5;

data test1;

do i = 1 to &row.;

output;

end;

run;

data test2;

set test1;

array econ econ_1 - econ_&col.;

array z z_1 - z_&col.;

do j = 1 to &col.;

if _n_ = 1 then econ{j} = rannor(1);

z{j} = econ{j} * i;

end;

retain econ_1 - econ_&col.;

run;

Re: arrays efficiency problem

ballardw — Fri, 14 Sep 2012 15:17:07 GMT

One minor point if you only have one input set then _N_ will only be one once so there is no reaso to include that test in the DO J= loop.

It sounds like you have a varying number of econ variables and you are sometimes creating extra variables that you don't want.

A quick and dirty approach would be to set COL using the data assuming you have a dataset with more meaningful ECON variables.

data _null_;

set have (obs=1);

array econ econ_: ;

x= dim(econ);

call symput('col',put(x,f5.0));

run;

Re: arrays efficiency problem

Weihong — Fri, 14 Sep 2012 15:41:36 GMT

"One minor point if you only have one input set then _N_ will only be one once so there is no reaso to include that test in the DO J= loop."

Not quite understand what you mean here. The input data test1 has 10 observations. And, _N_ will be used 5 times as there are 5 arrays being created.

"It sounds like you have a varying number of econ variables and you are sometimes creating extra variables that you don't want."

I do want to use those retained observations, but since they are all the same within same econ_*, it is redundent and it takes too much space. So I am thinking if I can write these 5 (&col.) econ_* as global variables, and retreat them when I need them.

Re: arrays efficiency problem

Weihong — Fri, 14 Sep 2012 19:32:54 GMT

Got help from SAS already. See below code. Appreciate your time.

%let col = 3;

Data _null_;

do i = 1 to &col.;

call symput ('econ_'||strip(i), rannor(1));

end;

run;

%macro test;

data test2;

set test1;

array z z_1 - z_&col.;

%do j = 1 %to &col.;

z{&j} = &&econ_&j. *i;

%end;

run;

%mend;

%test;

Re: arrays efficiency problem

ballardw — Fri, 14 Sep 2012 19:39:06 GMT

I thought your data was to simulate something in lieue of your actual data.

I do not know what you mean by redundant or what space you want to save.

If you do not want variables in the output data set but need them for calculations during the execution of the data set then drop them from the output.

data test2 (drop= econ_: ) ;

may be what your are looking for.

Re: arrays efficiency problem

Weihong — Fri, 14 Sep 2012 19:58:09 GMT

The real job envolved 1 million observation and 10 arrays, each array has a size of 5000. So by elimiating one array (econ, which is a random economic variable shared by all observations) helps to reduce the size dramatically. 'SAS help' provided me a solution (Please see 3. Re.) which is exactly what I mean to do.