topic Re: how to use horizontal direct product (HDIR) when the matrices have missing values? in SAS/IML Software and Matrix Computations

how to use horizontal direct product (HDIR) when the matrices have missing values?

ty004720 — Sun, 02 Jul 2017 15:09:26 GMT

Hello All,

There are two matrices with missing values. I try to do the horizontal direct product (HDIR) in SAS, but I am receiving the following error: "ERROR: Matrix c has not been set to a value." It seems that most matrix operators and functions in SAS do not support missing values. Could you please help me figure this out? Any assistance that can be provided is much appreciated.

Here is an example:

proc iml;
a = {1 .,
     2 4,
     3 6};
b = {0  2,
     .  1,
     0 -1};
c = hdir(a, b);
print a b c;

Here is the Log:

1 OPTIONS NONOTES NOSTIMER NOSOURCE NOSYNTAXCHECK;

60 proc iml;

NOTE: IML Ready

61 a = {1 .,

62 2 4,

63 3 6};

64 b = {0 2,

65 . 1,

66 0 -1};

67 c = hdir(a, b);

ERROR: (execution) Invalid argument or operand; contains missing values.

operation : HDIR at line 67 column 9

operands : a, b

a 3 rows 2 cols (numeric)

1 .

2 4

3 6

b 3 rows 2 cols (numeric)

0 2

. 1

0 -1

statement : ASSIGN at line 67 column 1

68 print a b c;

ERROR: Matrix c has not been set to a value.

statement : PRINT at line 68 column 1

70 OPTIONS NONOTES NOSTIMER NOSOURCE NOSYNTAXCHECK;

Re: how to use horizontal direct product (HDIR) when the matrices have missing values?

PaigeMiller — Sun, 02 Jul 2017 15:11:36 GMT

You'd have to write your own Function in IML to do what you want.

Re: how to use horizontal direct product (HDIR) when the matrices have missing values?

Rick_SAS — Sun, 02 Jul 2017 19:13:52 GMT

Although SAS/IML propagates missing values for elementwise operations, missing values do not propagate for vector and matrix operations. For a discussion and some workarounds, see "Matrix multiplication with missing values in SAS."

What is the application you are trying to compute? Are you creating a design matrix of interaction effects, or something else? Also, what answer do you EXPECT to get from your example?

Re: how to use horizontal direct product (HDIR) when the matrices have missing values?

ty004720 — Sun, 02 Jul 2017 19:33:03 GMT

Thanks very much for your reply. I am doing my research on interaction effects. We have a large dataset with 17 interested predictors. All the predictor variables have missing values. I need to create a new data set with all the 2-way interaction effects. I think horizontal direct product (HDIR) in SAS can help me create those interaction effects. First, I have successfully read all the predictor variables into a matrix (denoted by matrix_L18_Q34) in SAS. Then I try to do use the code "interaction_matrix_L18_Q34 = hdir(matrix_L18_Q34, matrix_L18_Q34)". However, it shows that "ERROR: (execution) Matrix has not been set to a value."

Here is the log:

1 OPTIONS NONOTES NOSTIMER NOSOURCE NOSYNTAXCHECK;

60 /* read all interested predictor variables (a to q) into matrix matrix_L18_Q34 */

61 proc iml;

NOTE: IML Ready

62 use L18_Q34;

63 read all var ("a":"q") into matrix_L18_Q34[colname=varNames];

64 print varNames;

65 print matrix_L18_Q34;

68 /*Horizontal Direct Product*/

NOTE: Exiting IML.

NOTE: PROCEDURE IML used (Total process time):

real time 2.64 seconds

user cpu time 2.61 seconds

system cpu time 0.00 seconds

memory 4135.40k

OS Memory 32168.00k

Timestamp 07/02/2017 07:29:14 PM

Step Count 24 Switch Count 50

Page Faults 0

Page Reclaims 847

Page Swaps 0

Voluntary Context Switches 157

Involuntary Context Switches 272

Block Input Operations 0

Block Output Operations 1440

69 proc iml;

NOTE: IML Ready

70 interaction_matrix_L18_Q34 = hdir(matrix_L18_Q34, matrix_L18_Q34);

ERROR: (execution) Matrix has not been set to a value.

operation : HDIR at line 70 column 34

operands : matrix_L18_Q34, matrix_L18_Q34

matrix_L18_Q34 0 row 0 col (type ?, size 0)

statement : ASSIGN at line 70 column 1

71 print interaction_matrix_L18_Q34;

ERROR: Matrix interaction_matrix_L18_Q34 has not been set to a value.

statement : PRINT at line 71 column 1

73 OPTIONS NONOTES NOSTIMER NOSOURCE NOSYNTAXCHECK;

Re: how to use horizontal direct product (HDIR) when the matrices have missing values?

ty004720 — Sun, 02 Jul 2017 19:34:40 GMT

I read "Matrix multiplication with missing values in SAS" before. However, it does not answer my questions on HDIR.

Re: how to use horizontal direct product (HDIR) when the matrices have missing values?

Rick_SAS — Mon, 03 Jul 2017 14:17:43 GMT

As Paige said, use the definition to write your own function. You didn't say what you expect the result to be, but I assume you want to propagate the missing like this:

proc iml;
start HDir2(a, b);
   c = j(nrow(a), ncol(a)*ncol(b), .);
   do i = 1 to ncol(a);
      startCol = (i-1)*ncol(b) + 1;
      endCol = i*ncol(b);
      c[ , startCol:endCol] = a[ ,i] # b;
   end;
   return c;
finish;

a = {1 .,
     2 4,
     3 6};
b = {0  2,
     .  1,
     0 -1};
c = HDir2(a, b);
print c;

Re: how to use horizontal direct product (HDIR) when the matrices have missing values?

ty004720 — Mon, 03 Jul 2017 14:08:34 GMT

Thanks very much. You answer really helps.

Re: how to use horizontal direct product (HDIR) when the matrices have missing values?

Ksharp — Mon, 03 Jul 2017 14:11:31 GMT

How about this one ?

proc iml;
a = {1 .,
     2 4,
     3 6};
b = {0  2,
     .  1,
     0 -1};
aa=a;
bb=b;
aa[loc(a=.)]=0;
bb[loc(b=.)]=0;
c = hdir(aa, bb);

a1=(a=.);
b1=(b=.);

a2=j(nrow(a),ncol(a));
b2=j(nrow(b),ncol(b));

a1b2=hdir(a1,b2);
a2b1=hdir(a2,b1); 

c[loc(a1b2=1)]=.;
c[loc(a2b1=1)]=.;
print c;

quit;

SAS Output

c
0	2	.	.
.	2	.	4
0	-3	0	-6

Re: how to use horizontal direct product (HDIR) when the matrices have missing values?

IanWakeling — Mon, 03 Jul 2017 14:53:27 GMT

Assuming the matrices you are working with are not large, it might be easier to calculate more than necessary and then throw away the unwanted parts.

proc iml;
a = {1 .,
     2 4,
     3 6};
b = {0  2,
     .  1,
     0 -1};

n = nrow( a );
c = (a @ b)[ do(1, n#n, n+1), ];

print a b c;

Re: how to use horizontal direct product (HDIR) when the matrices have missing values?

ty004720 — Wed, 05 Jul 2017 13:14:04 GMT

Thanks very much. I like your ideas.