topic Re: IML code in SAS/IML Software and Matrix Computations

IML code

JackSmith — Thu, 01 Oct 2009 22:23:26 GMT

I want to reduce this code so that it can run fast and use less memory.

Please if your too good with IML can you please help me.

Thanks
Jacksmith

proc iml;
use hedonic2;
read all var {lsale_amt} into y;
read all var {tract_zip} into tract;
read all var {ltv bath bed age gla1000 lot1000 gla2 lot2 bed2 bath2 age2 tax
MONTH1 MONTH2 MONTH3 MONTH4 MONTH5 MONTH6 MONTH7 MONTH8 MONTH9 MONTH10 MONTH11 MONTH12} into x;
use tract_mean; read all var {count} into ncount;
use wnor; read all var _all_ into wnor;
use outest_step1;
read all var {a1 a2 a3 a4 a5 a6 a7 a8 a9 a10 a11 a12 a13 a14 a15 a16 a17 a18 a19 a20
a21 a22 a23 a24 a25 } into beta;

nn=nrow(y);
beta1=beta`;
beta=beta1[1:24];
lamda=beta1[25];

i=i(nn);
H2=wnor*inv(i-lamda*wnor)*x*beta;
H=H2||tract;

create H from H; append from H; close H;
quit;
options nomprint nonotes NOPRINTMSGLIST NOSOURCE NOSOURCE2;

data hedonic2;
merge hedonic2 H(rename=(col1=H col2=tract_zip));
by tract_zip;
run;

proc model data=hedonic2 noprint
;
endogenous wy;
lsale_amt=
a1* ltv +
a2* bath +
a3* bed +
a4* age +
a5* gla1000 +
a6* lot1000 +
a7* gla2 +
a8* lot2 +
a9* bed2 +
a10* bath2 +
a11* age2 +
a12* tax +

a13* month1 +
a14* month2 +
a15* month3 +
a16* month4 +
a17* month5 +
a18* month6 +
a19* month7 +
a20* month8 +
a21* month9 +
a22* month10 +
a23* month11 +
a24* month12 +

a25* wy
;
fit lsale_amt /2sls out=out outresid outpredict outest=outest outcov outs=sigma hccme=1;
instruments
ltv bath bed age gla1000 lot1000 gla2 lot2 bed2 bath2 age2 tax
month1-month12
h
/noint;
run;
quit;

Re: IML code

Hutch_sas — Fri, 02 Oct 2009 15:43:25 GMT

One immediate change you can make for efficiency and accuracy is to use the SOLVE function instead of taking the inverse:

instead of

H2=wnor*inv(i-lamda*wnor)*x*beta;

use

H2=wnor*solve( i-lamda*wnor,x)*beta;

Re: IML code

Rick_SAS — Fri, 02 Oct 2009 17:50:44 GMT

Remember, readers, sending code that can actually be run increases the chances of getting an answer to your problem.

Hutch has the right idea, but you don't want to solve with a RHS of x. Form the quantity z=x*beta, which is a vector, and then use SOLVE.

You don't give the dimensions of this problem, which makes it difficult to reproduce what you are doing. However, I'm guessing that wnor is a row vector? Then here is some code similar to yours that we can all actually run:

proc iml;
nn=100;
p = 24;
x = rannor( j(nn,p,1) );
beta=T(p:1);
lambda=1;
wnor = 1:nn;

I=I(nn);
z = x*beta;
H2=wnor*inv(I-lambda*wnor)*z;
print H2;

As Hutch says, the following code improves the efficiency and cuts down on a matrix multiplication:
/* Let q = inv(I-lambda*wnor)*(x*beta).
Then q = solve( I-lambda*wnor, x*beta )
*/
A = I-lambda*wnor;
q = solve( A, z );
H3 = wnor*q;
print H3;

If memory is really tight, you can also eliminate the explicity construction of the identity matrix:

/* even less memory...do not explicitly form identity */
A = repeat( -lambda*wnor, nn );
do i = 1 to nn;
A[i,i] = 1 + A[i,i];
end;
free x; /* you don't don't need this big matrix anymore */
q = solve( A, z );
H4 = wnor*q;
print H4;

Re: IML code

JackSmith — Mon, 05 Oct 2009 21:08:48 GMT

Thank you for your input. Message was edited by: JackSmith