topic Re: Proc IML missing values question in SAS/IML Software and Matrix Computations

Proc IML missing values question

trekvana — Wed, 08 Sep 2010 22:24:37 GMT

Hello all,

Here is my basic scenario.

Lets say I read in a 2x2 matrix from a SAS data set say, A=[1,2 ; 3,4] and I have to multiply it into a 3 dimensional column vector with a missing value say, x=(4, . ,6)'.

Now I know that proc IML doesnt like missing values so I have to first remove the missing value from x then I can do A*x and now I get a 2x1 column vector.

What I want to do now is reinsert the missing value into its original position as with the x vector. So in our case y=A*x=[16,36] and I want y to become y=[16, . , 36].

This was just a basic example. My x vectors may have multiple missing values at different indices. My A matrix will always be the dimension of the truncated x vector so I dont need to worry about my A matrix. All I need to do is truncate the missing values from my x vector, do the matrix multiplication, and then add the missing value character (.) in y where it initially was on x.

I am not that proficient in coding yet so any complete or semi-complete code would help.

George

Re: Proc IML missing values question

Rick_SAS — Thu, 09 Sep 2010 10:03:34 GMT

I can tell that you have a MATLAB background. Welcome to IML!

In IML, use a space to separate columns and use a comma to indicate a new row. So your data might look like this:

A = {1 2 3, 4 5 6, 7 8 9};
x = {1, ., ., 4, ., 6, .};

The function you want is the LOC function, which can find the indices of x that are missing (or nonmissing):

NMidx = loc(x ^= .); /** non-missing index **/
v = x[NMidx]; /** subset to form nonmissing values of x **/

y = repeat(., nrow(x)); /** fill answer vector with missing values **/
y[NMidx] = A*v; /** put answer in correct locations **/
print y;

Re: Proc IML missing values question

trekvana — Thu, 09 Sep 2010 16:53:58 GMT

Rick-

Suppose now that I had a matrix instead of a vector, let say A={1 2; 3 4} and B={1 . 3; . 1 2}. Can I still use the loc function to find the indices of the missing values for the B matrix?

So I would need to get B1={1 3;1 2}, then do A*B1 and then augment the B1 matrix with corresponding missing values. Here is what I thought of. I assume there is a faster way?

proc iml;
a={1 2,3 4};
b={1 . 3,. 1 2};
nm=loc(b^=.);
b1=shape(b[loc(b^=.)],nrow(a),ncol(a));
y=shape(a*b1,ncol(nm));
y1=repeat(.,nrow(b)*ncol(b));
y1[nm]=y;
y1=shape(y1,nrow(b),ncol(b));
print a,b,nm,b1,y,y1;
quit;

Message was edited by: trekvana

Re: Proc IML missing values question

Rick_SAS — Thu, 09 Sep 2010 18:14:04 GMT

That's not bad. I think I'd shape y1 to be a matrix and get rid of y:

nm=loc(b^=.);
b1=shape(b[loc(b^=.)],nrow(a),ncol(a));
y1=repeat(.,nrow(b),ncol(b));
y1[nm]=a*b1;
print nm,b1,y1;

Can anyone else come up with a better technique?

Incidentally, I discuss handling missing values in Ch. 3 and Ch. 11 of my forthcoming book on IML. This problem you've posed is a variation of the problems and solutions in those chapters.

Re: Proc IML missing values question

trekvana — Fri, 10 Sep 2010 00:57:43 GMT

Rick-

when is your book slated to be released? Do you have a title for it already (or at least something on how to find it when it comes out). I look forward to it.

Thanks for the help

George

Re: Proc IML missing values question

trekvana — Fri, 10 Sep 2010 00:59:36 GMT

Rick-

when is your book slated to be released? Do you have a title for it already (or at least something on how to find it when it comes out). I look forward to it.

Thanks for the help.

George

EDIT: A quick search and I believe I found it
http://support.sas.com/publishing/authors/wicklin.html

Re: Proc IML missing values question

Rick_SAS — Fri, 10 Sep 2010 12:33:51 GMT

That's it. I was told the book would be available "late October to early November, 2010."