topic Re: 95% confidence intervals for the eigenvalues in SAS/IML Software and Matrix Computations

95% confidence intervals for the eigenvalues

lakshmishree — Fri, 09 Oct 2020 13:10:41 GMT

Hey all,

I need to find 95% confidence intervals of correlation matrix for the first to fifth eigenvalues of a dataset (n=178). Where, I tried giving it using proc iml as below:

proc iml;
reset print;
eig=eigval(THC);
E=eigvec(THC);
alpha=0.05;
zright=quantile("Normal", 1-alpha/2);
CI_1_lower=eig[1]/(1+zright*sqrt(2/178));
CI_1_upper=eig[1]/(1-zright*sqrt(2/178));
CI_2_lower=eig[2]/(1+zright*sqrt(2/178));
CI_2_upper=eig[2]/(1-zright*sqrt(2/178));
CI_3_lower=eig[3]/(1+zright*sqrt(2/178));
CI_3_upper=eig[3]/(1-zright*sqrt(2/178));
CI_4_lower=eig[4]/(1+zright*sqrt(2/178));
CI_4_upper=eig[4]/(1-zright*sqrt(2/178));
CI_5_lower=eig[5]/(1+zright*sqrt(2/178));
CI_5_upper=eig[5]/(1-zright*sqrt(2/178));

I get only the following output but not intervals.

alpha 1 row 1 col (numeric)

0.05

zright 1 row 1 col (numeric)

1.959964

Can anyone tell me where am i going wrong?

Thanks,

lakshmi

Re: 95% confidence intervals for the eigenvalues

SAS_Rob — Fri, 09 Oct 2020 17:41:01 GMT

Do you have a SAS LOG you could post to see the code in context?

Re: 95% confidence intervals for the eigenvalues

Rick_SAS — Fri, 09 Oct 2020 17:42:07 GMT

Whenever you submit a SAS program, you should look at the log to make sure there are no errors. In this case, the log displays

3 eig=eigval(THC);
ERROR: (execution) Matrix has not been set to a value.

operation : EIGVAL at line 3 column 11
operands : THC

THC 0 row 0 col (type ?, size 0)

The error message tells you that the THC matrix (which is the argument to the EIGVAL function) is not defined. Before you try to use it, you need to read it from a data set or define it manually:

THC = {5 4 3 2 1 ,
       4 5 4 3 2 ,
       3 4 5 4 3 ,
       2 3 4 5 4 ,
       1 2 3 4 5 };

Re: 95% confidence intervals for the eigenvalues

Jenkos — Sat, 10 Oct 2020 07:01:58 GMT

I'm also trying to do this. I've added read all var _all_ into x; to read the THC dataset into a matrix as below, how I'm getting the following errors:

1 OPTIONS NONOTES NOSTIMER NOSOURCE NOSYNTAXCHECK;

73 proc iml;

NOTE: IML Ready

74 use THC;

75 reset print;

76 read all var _all_ into x;

77 eig=eigval(x);

ERROR: (execution) Matrix should be square.

operation : EIGVAL at line 77 column 11

operands : x

x 178 rows 14 cols (numeric)

statement : ASSIGN at line 77 column 1

78 e_vec=eigvec(x);

ERROR: (execution) Matrix should be square.

operation : EIGVEC at line 78 column 13

operands : x

x 178 rows 14 cols (numeric)

statement : ASSIGN at line 78 column 1

79 CI_1_lower=eig[1]/(1+1.645*sqrt(2/178));

ERROR: (execution) Matrix has not been set to a value.

proc iml;
use THC;
reset print;
read all var _all_ into x;
eig=eigval(x);
e_vec=eigvec(x);
CI_1_lower=eig[1]/(1+1.645*sqrt(2/178));
CI_1_upper=eig[1]/(1-1.645*sqrt(2/178));
CI_2_lower=eig[2]/(1+1.645*sqrt(2/178));
CI_2_upper=eig[2]/(1-1.645*sqrt(2/178));
CI_3_lower=eig[3]/(1+1.645*sqrt(2/178));
CI_3_upper=eig[3]/(1-1.645*sqrt(2/178));
CI_4_lower=eig[4]/(1+1.645*sqrt(2/178));
CI_4_upper=eig[4]/(1-1.645*sqrt(2/178));
CI_5_lower=eig[5]/(1+1.645*sqrt(2/178));
CI_5_upper=eig[5]/(1-1.645*sqrt(2/178));

Re: 95% confidence intervals for the eigenvalues

Rick_SAS — Sat, 10 Oct 2020 09:23:32 GMT

Mathematically, you can only find eigenvalues/eigenvectors for a square matrix. This is what the error message is telling you. Apparently, you are passing in a 178 x 14 matrix.

ERROR: (execution) Matrix should be square.

operation : EIGVAL at line 77 column 11

operands : x

x 178 rows 14 cols (numeric)

Re: 95% confidence intervals for the eigenvalues

Rick_SAS — Sat, 10 Oct 2020 09:56:39 GMT

In multivariate statistics (such as principal component analysis), it is useful to find the eigenvalues of a correlation or a covariance matrix. Is that what you are trying to do? If so, use

R = corr(X);

S = cov(X);

to get the correlation or covariance matrix and then analyze the eigenvalues of R or S.

Re: 95% confidence intervals for the eigenvalues

lakshmishree — Sun, 11 Oct 2020 09:47:36 GMT

you are right, I am here trying to find the eigenvalues of a correlation matrix which is 179*14 matrix.

Tried running following command as you said, but got same errors.

proc princomp data=THC;
run;

/* 95% confidence intervals for the first to fifth eigenvalues */
proc iml;
reset print;
R=corr(THC);
eig=eigval(R);
E=eigvec(R);
alpha=0.05;
zright=quantile("Normal", 1-alpha/2);
CI_1_lower=eig[1]/(1+zright*sqrt(2/179));
CI_1_upper=eig[1]/(1-zright*sqrt(2/179));
CI_2_lower=eig[2]/(1+zright*sqrt(2/179));
CI_2_upper=eig[2]/(1-zright*sqrt(2/179));
CI_3_lower=eig[3]/(1+zright*sqrt(2/179));
CI_3_upper=eig[3]/(1-zright*sqrt(2/179));
CI_4_lower=eig[4]/(1+zright*sqrt(2/179));
CI_4_upper=eig[4]/(1-zright*sqrt(2/179));
CI_5_lower=eig[5]/(1+zright*sqrt(2/179));
CI_5_upper=eig[5]/(1-zright*sqrt(2/179));

ERROR: (execution) Matrix has not been set to a value.

operation : CORR at line 75 column 7

operands : THC

THC 0 row 0 col (type ?, size 0)

statement : ASSIGN at line 75 column 1

77 OPTIONS NONOTES NOSTIMER NOSOURCE NOSYNTAXCHECK;

Re: 95% confidence intervals for the eigenvalues

lakshmishree — Sun, 11 Oct 2020 09:53:07 GMT

ERROR: (execution) Matrix has not been set to a value.

operation : CORR at line 75 column 7
operands : THC

THC 0 row 0 col (type ?, size 0)

statement : ASSIGN at line 75 column 1
76
77 OPTIONS NONOTES NOSTIMER NOSOURCE NOSYNTAXCHECK;
89

Re: 95% confidence intervals for the eigenvalues

Rick_SAS — Sun, 11 Oct 2020 10:04:43 GMT

Same problem as last time: You aren't reading the data. You have to read the data into IML before you can use it.

Try this:

proc iml;
use THC;
read all var _all_ into X;
close;

R = corr(X);
eig=eigval(R);
do i = 1 to 5;
  CI_lower=eig[i]/(1+1.645*sqrt(2/178));
  CI_upper=eig[i]/(1-1.645*sqrt(2/178));
  print i (eig[i])[L="eig"] CI_Lower CI_Upper;
end;

Re: 95% confidence intervals for the eigenvalues

lakshmishree — Mon, 12 Oct 2020 07:00:03 GMT

Now i am able to print the Confidence interval values. Whereas, eigenvalues seems to be different from what i obtained. i Shall copy that code and the output i obtained from both:

SAS code:
proc princomp data=THC;
var chem1 chem2 chem3 chem4 chem5 chem6 chem7 chem8 chem9 chem10 chem11 chem12 chem13;
run;

SAS output:
Eigenvalue
1 4.70585025
2 2.49697373
3 1.44607197
4 0.91897392
5 0.85322818

Whereas when i am trying to run your code i obtain different eigenvalues which is actually of correlation matrix again, since i have "varieties" as another variable in the data and by reading all here, i assume it's including even that variable in it. Could you help me with what changes i would read only selected variables? that is if i want to read only variables : chem1 chem2 chem3 chem4 chem5 chem6 chem7 chem8 chem9 chem10 chem11 chem12 chem13

Re: 95% confidence intervals for the eigenvalues

IanWakeling — Mon, 12 Oct 2020 08:24:42 GMT

You can modify the READ statement like this:

  use THC;
  read all var ('chem1':'chem13') into X;
  close THC;

Re: 95% confidence intervals for the eigenvalues

lakshmishree — Tue, 13 Oct 2020 08:07:00 GMT

Thank you, it worked