Rounding issues

mgx · Posted 04-22-2024 08:14 AM

Dear SAS community,

I am working with an imputed dataset and first performing a sandwich estimator to compute the within-imputation variances on each imputation before combining the results. My proc iml code for the sandwich estimator looks as follows:

%macro sandwich_imput(i=);
data hessian;
set mvp.hessian_imput;
where _imputation_=&i;
drop row Parameter _imputation_;
run;
data gradient;
set mvp.grad_imput;
where _imputation_=&i;
drop observation _imputation_;
run;
proc iml;
use hessian; read all into J; close hessian;
use gradient; read all into G; close gradient;
use parms; read all into P; close parms;
K=t(G)*(G);
Sigma=round(inv(J)*K*inv(J),0.001);
create Sigma from Sigma;append from Sigma;
quit;
data cov;
set Sigma cov;
run;
%mend sandwich_imput;

After adding the appropriate extra variables, I use the cov dataset in PROC MIANALYZE. However, due to rounding errors I get the following error:

ERROR: Within-imputation COVB matrix is not symmetric for _Imputation_= 1 in the input COVB= data set.

The rounded variance-covariance matrix is symmetric, but it seems that SAS internally stores the non-rounded matrix and uses that matrix in PROC MIANALYZE. Is there any way to circumvent this problem?

ballardw · Posted 04-22-2024 09:42 AM

Please do not separate error messages from the code that generates them. Include, from the LOG copy the code and all the notes as well as the errors. Then paste all of that into a text box on the forum.

Often those other messages have diagnostic information. Also without the code we have no clue what options you may have invoked that might relate to that specific error.

Also be aware that this style of coding where one (or more) of the data sets on a SET or MERGE statement appear on the DATA statement is somewhat fragile as the source data set is completely replaced by the on the Data statement.

data cov;
set Sigma cov;
run;

And even more problematic buried in a macro as it is easy to forget that you are doing so every time you run the macro. So if had to run that macro a couple of times such as debugging logic you have made multiple additions to the Cov data set.

mgx · Posted 04-22-2024 09:45 AM

Here you can find the full log:

28 data cov;
29 run;

NOTE: The data set WORK.COV has 1 observations and 0 variables.
NOTE: DATA statement used (Total process time):
real time 0.01 seconds
cpu time 0.00 seconds

30 %macro sandwich_imput(i=);
31 data hessian;
32 set mvp.hessian_imput;
33 where _imputation_=&i;
34 drop row Parameter _imputation_;
35 run;
36 data gradient;
37 set mvp.grad_imput;
38 where _imputation_=&i;
39 drop observation _imputation_;
40 run;
41 proc iml;
42 use hessian; read all into J; close hessian;
43 use gradient; read all into G; close gradient;
44 use parms; read all into P; close parms;
45 K=t(G)*(G);
46 Sigma=round(inv(J)*K*inv(J),0.001);
47 create Sigma from Sigma;append from Sigma;
48 quit;
2 The SAS System 09:59 Monday, April 22, 2024

49 data cov;
50 set Sigma cov;
51 run;
52 data cov;
53 set cov;
54 where col1 is not missing;
55 run;
56 %mend sandwich_imput;
57 %sandwich_imput(i=1)

NOTE: There were 15 observations read from the data set MVP.HESSIAN_IMPUT.
WHERE _imputation_=1;
NOTE: The data set WORK.HESSIAN has 15 observations and 15 variables.
NOTE: DATA statement used (Total process time):
real time 0.01 seconds
cpu time 0.00 seconds

NOTE: There were 1338 observations read from the data set MVP.GRAD_IMPUT.
WHERE _imputation_=1;
NOTE: The data set WORK.GRADIENT has 1338 observations and 15 variables.
NOTE: DATA statement used (Total process time):
real time 0.02 seconds
cpu time 0.00 seconds

NOTE: IML Ready
NOTE: Exiting IML.
NOTE: The data set WORK.SIGMA has 15 observations and 15 variables.
NOTE: PROCEDURE IML used (Total process time):
real time 0.08 seconds
cpu time 0.00 seconds

NOTE: There were 15 observations read from the data set WORK.SIGMA.
NOTE: There were 1 observations read from the data set WORK.COV.
NOTE: The data set WORK.COV has 16 observations and 15 variables.
NOTE: DATA statement used (Total process time):
real time 0.00 seconds
cpu time 0.00 seconds

NOTE: There were 15 observations read from the data set WORK.COV.
WHERE col1 is not null;
NOTE: The data set WORK.COV has 15 observations and 15 variables.
NOTE: DATA statement used (Total process time):
real time 0.00 seconds
cpu time 0.00 seconds

58 %sandwich_imput(i=2)

NOTE: There were 15 observations read from the data set MVP.HESSIAN_IMPUT.
WHERE _imputation_=2;
NOTE: The data set WORK.HESSIAN has 15 observations and 15 variables.
3 The SAS System 09:59 Monday, April 22, 2024

NOTE: DATA statement used (Total process time):
real time 0.00 seconds
cpu time 0.01 seconds

NOTE: There were 1338 observations read from the data set MVP.GRAD_IMPUT.
WHERE _imputation_=2;
NOTE: The data set WORK.GRADIENT has 1338 observations and 15 variables.
NOTE: DATA statement used (Total process time):
real time 0.00 seconds
cpu time 0.00 seconds

NOTE: IML Ready
NOTE: Exiting IML.
NOTE: The data set WORK.SIGMA has 15 observations and 15 variables.
NOTE: PROCEDURE IML used (Total process time):
real time 0.01 seconds
cpu time 0.00 seconds

NOTE: There were 15 observations read from the data set WORK.SIGMA.
NOTE: There were 15 observations read from the data set WORK.COV.
NOTE: The data set WORK.COV has 30 observations and 15 variables.
NOTE: DATA statement used (Total process time):
real time 0.02 seconds
cpu time 0.00 seconds

NOTE: There were 30 observations read from the data set WORK.COV.
WHERE col1 is not null;
NOTE: The data set WORK.COV has 30 observations and 15 variables.
NOTE: DATA statement used (Total process time):
real time 0.00 seconds
cpu time 0.00 seconds

59 %sandwich_imput(i=3)

NOTE: There were 15 observations read from the data set MVP.HESSIAN_IMPUT.
WHERE _imputation_=3;
NOTE: The data set WORK.HESSIAN has 15 observations and 15 variables.
NOTE: DATA statement used (Total process time):
real time 0.00 seconds
cpu time 0.00 seconds

NOTE: There were 1338 observations read from the data set MVP.GRAD_IMPUT.
WHERE _imputation_=3;
NOTE: The data set WORK.GRADIENT has 1338 observations and 15 variables.
NOTE: DATA statement used (Total process time):
real time 0.00 seconds
cpu time 0.00 seconds

4 The SAS System 09:59 Monday, April 22, 2024

NOTE: IML Ready
NOTE: Exiting IML.
NOTE: The data set WORK.SIGMA has 15 observations and 15 variables.
NOTE: PROCEDURE IML used (Total process time):
real time 0.01 seconds
cpu time 0.00 seconds

NOTE: There were 15 observations read from the data set WORK.SIGMA.
NOTE: There were 30 observations read from the data set WORK.COV.
NOTE: The data set WORK.COV has 45 observations and 15 variables.
NOTE: DATA statement used (Total process time):
real time 0.00 seconds
cpu time 0.00 seconds

NOTE: There were 45 observations read from the data set WORK.COV.
WHERE col1 is not null;
NOTE: The data set WORK.COV has 45 observations and 15 variables.
NOTE: DATA statement used (Total process time):
real time 0.00 seconds
cpu time 0.00 seconds

60 %sandwich_imput(i=4)

NOTE: There were 15 observations read from the data set MVP.HESSIAN_IMPUT.
WHERE _imputation_=4;
NOTE: The data set WORK.HESSIAN has 15 observations and 15 variables.
NOTE: DATA statement used (Total process time):
real time 0.00 seconds
cpu time 0.00 seconds

NOTE: There were 1338 observations read from the data set MVP.GRAD_IMPUT.
WHERE _imputation_=4;
NOTE: The data set WORK.GRADIENT has 1338 observations and 15 variables.
NOTE: DATA statement used (Total process time):
real time 0.01 seconds
cpu time 0.00 seconds

NOTE: IML Ready
NOTE: Exiting IML.
NOTE: The data set WORK.SIGMA has 15 observations and 15 variables.
NOTE: PROCEDURE IML used (Total process time):
real time 0.01 seconds
cpu time 0.00 seconds

NOTE: There were 15 observations read from the data set WORK.SIGMA.
NOTE: There were 45 observations read from the data set WORK.COV.
NOTE: The data set WORK.COV has 60 observations and 15 variables.
5 The SAS System 09:59 Monday, April 22, 2024

NOTE: DATA statement used (Total process time):
real time 0.00 seconds
cpu time 0.00 seconds

NOTE: There were 60 observations read from the data set WORK.COV.
WHERE col1 is not null;
NOTE: The data set WORK.COV has 60 observations and 15 variables.
NOTE: DATA statement used (Total process time):
real time 0.00 seconds
cpu time 0.00 seconds

61 %sandwich_imput(i=5)

NOTE: There were 15 observations read from the data set MVP.HESSIAN_IMPUT.
WHERE _imputation_=5;
NOTE: The data set WORK.HESSIAN has 15 observations and 15 variables.
NOTE: DATA statement used (Total process time):
real time 0.00 seconds
cpu time 0.00 seconds

NOTE: There were 1338 observations read from the data set MVP.GRAD_IMPUT.
WHERE _imputation_=5;
NOTE: The data set WORK.GRADIENT has 1338 observations and 15 variables.
NOTE: DATA statement used (Total process time):
real time 0.00 seconds
cpu time 0.01 seconds

NOTE: IML Ready
NOTE: Exiting IML.
NOTE: The data set WORK.SIGMA has 15 observations and 15 variables.
NOTE: PROCEDURE IML used (Total process time):
real time 0.01 seconds
cpu time 0.00 seconds

NOTE: There were 15 observations read from the data set WORK.SIGMA.
NOTE: There were 60 observations read from the data set WORK.COV.
NOTE: The data set WORK.COV has 75 observations and 15 variables.
NOTE: DATA statement used (Total process time):
real time 0.01 seconds
cpu time 0.00 seconds

NOTE: There were 75 observations read from the data set WORK.COV.
WHERE col1 is not null;
NOTE: The data set WORK.COV has 75 observations and 15 variables.
NOTE: DATA statement used (Total process time):
real time 0.00 seconds
cpu time 0.00 seconds

6 The SAS System 09:59 Monday, April 22, 2024

62 %sandwich_imput(i=6)

NOTE: There were 15 observations read from the data set MVP.HESSIAN_IMPUT.
WHERE _imputation_=6;
NOTE: The data set WORK.HESSIAN has 15 observations and 15 variables.
NOTE: DATA statement used (Total process time):
real time 0.00 seconds
cpu time 0.00 seconds

NOTE: There were 1338 observations read from the data set MVP.GRAD_IMPUT.
WHERE _imputation_=6;
NOTE: The data set WORK.GRADIENT has 1338 observations and 15 variables.
NOTE: DATA statement used (Total process time):
real time 0.00 seconds
cpu time 0.00 seconds

NOTE: IML Ready
NOTE: Exiting IML.
NOTE: The data set WORK.SIGMA has 15 observations and 15 variables.
NOTE: PROCEDURE IML used (Total process time):
real time 0.01 seconds
cpu time 0.00 seconds

NOTE: There were 15 observations read from the data set WORK.SIGMA.
NOTE: There were 75 observations read from the data set WORK.COV.
NOTE: The data set WORK.COV has 90 observations and 15 variables.
NOTE: DATA statement used (Total process time):
real time 0.00 seconds
cpu time 0.00 seconds

NOTE: There were 90 observations read from the data set WORK.COV.
WHERE col1 is not null;
NOTE: The data set WORK.COV has 90 observations and 15 variables.
NOTE: DATA statement used (Total process time):
real time 0.00 seconds
cpu time 0.00 seconds

63 %sandwich_imput(i=7)

NOTE: There were 15 observations read from the data set MVP.HESSIAN_IMPUT.
WHERE _imputation_=7;
NOTE: The data set WORK.HESSIAN has 15 observations and 15 variables.
NOTE: DATA statement used (Total process time):
real time 0.00 seconds
cpu time 0.01 seconds

NOTE: There were 1338 observations read from the data set MVP.GRAD_IMPUT.
7 The SAS System 09:59 Monday, April 22, 2024

WHERE _imputation_=7;
NOTE: The data set WORK.GRADIENT has 1338 observations and 15 variables.
NOTE: DATA statement used (Total process time):
real time 0.00 seconds
cpu time 0.00 seconds

NOTE: IML Ready
NOTE: Exiting IML.
NOTE: The data set WORK.SIGMA has 15 observations and 15 variables.
NOTE: PROCEDURE IML used (Total process time):
real time 0.01 seconds
cpu time 0.00 seconds

NOTE: There were 15 observations read from the data set WORK.SIGMA.
NOTE: There were 90 observations read from the data set WORK.COV.
NOTE: The data set WORK.COV has 105 observations and 15 variables.
NOTE: DATA statement used (Total process time):
real time 0.01 seconds
cpu time 0.00 seconds

NOTE: There were 105 observations read from the data set WORK.COV.
WHERE col1 is not null;
NOTE: The data set WORK.COV has 105 observations and 15 variables.
NOTE: DATA statement used (Total process time):
real time 0.00 seconds
cpu time 0.00 seconds

64 %sandwich_imput(i=8)

NOTE: There were 15 observations read from the data set MVP.HESSIAN_IMPUT.
WHERE _imputation_=8;
NOTE: The data set WORK.HESSIAN has 15 observations and 15 variables.
NOTE: DATA statement used (Total process time):
real time 0.00 seconds
cpu time 0.01 seconds

NOTE: There were 1338 observations read from the data set MVP.GRAD_IMPUT.
WHERE _imputation_=8;
NOTE: The data set WORK.GRADIENT has 1338 observations and 15 variables.
NOTE: DATA statement used (Total process time):
real time 0.00 seconds
cpu time 0.00 seconds

NOTE: IML Ready
NOTE: Exiting IML.
NOTE: The data set WORK.SIGMA has 15 observations and 15 variables.
NOTE: PROCEDURE IML used (Total process time):
real time 0.01 seconds
cpu time 0.00 seconds
8 The SAS System 09:59 Monday, April 22, 2024

NOTE: There were 15 observations read from the data set WORK.SIGMA.
NOTE: There were 105 observations read from the data set WORK.COV.
NOTE: The data set WORK.COV has 120 observations and 15 variables.
NOTE: DATA statement used (Total process time):
real time 0.01 seconds
cpu time 0.00 seconds

NOTE: There were 120 observations read from the data set WORK.COV.
WHERE col1 is not null;
NOTE: The data set WORK.COV has 120 observations and 15 variables.
NOTE: DATA statement used (Total process time):
real time 0.00 seconds
cpu time 0.00 seconds

65 %sandwich_imput(i=9)

NOTE: There were 15 observations read from the data set MVP.HESSIAN_IMPUT.
WHERE _imputation_=9;
NOTE: The data set WORK.HESSIAN has 15 observations and 15 variables.
NOTE: DATA statement used (Total process time):
real time 0.00 seconds
cpu time 0.00 seconds

NOTE: There were 1338 observations read from the data set MVP.GRAD_IMPUT.
WHERE _imputation_=9;
NOTE: The data set WORK.GRADIENT has 1338 observations and 15 variables.
NOTE: DATA statement used (Total process time):
real time 0.00 seconds
cpu time 0.00 seconds

NOTE: IML Ready
NOTE: Exiting IML.
NOTE: The data set WORK.SIGMA has 15 observations and 15 variables.
NOTE: PROCEDURE IML used (Total process time):
real time 0.02 seconds
cpu time 0.03 seconds

NOTE: There were 15 observations read from the data set WORK.SIGMA.
NOTE: There were 120 observations read from the data set WORK.COV.
NOTE: The data set WORK.COV has 135 observations and 15 variables.
NOTE: DATA statement used (Total process time):
real time 0.01 seconds
cpu time 0.00 seconds

NOTE: There were 135 observations read from the data set WORK.COV.
9 The SAS System 09:59 Monday, April 22, 2024

WHERE col1 is not null;
NOTE: The data set WORK.COV has 135 observations and 15 variables.
NOTE: DATA statement used (Total process time):
real time 0.01 seconds
cpu time 0.00 seconds

66 %sandwich_imput(i=10)

NOTE: There were 15 observations read from the data set MVP.HESSIAN_IMPUT.
WHERE _imputation_=10;
NOTE: The data set WORK.HESSIAN has 15 observations and 15 variables.
NOTE: DATA statement used (Total process time):
real time 0.00 seconds
cpu time 0.00 seconds

NOTE: There were 1338 observations read from the data set MVP.GRAD_IMPUT.
WHERE _imputation_=10;
NOTE: The data set WORK.GRADIENT has 1338 observations and 15 variables.
NOTE: DATA statement used (Total process time):
real time 0.00 seconds
cpu time 0.00 seconds

NOTE: IML Ready
NOTE: Exiting IML.
NOTE: The data set WORK.SIGMA has 15 observations and 15 variables.
NOTE: PROCEDURE IML used (Total process time):
real time 0.01 seconds
cpu time 0.00 seconds

NOTE: There were 15 observations read from the data set WORK.SIGMA.
NOTE: There were 135 observations read from the data set WORK.COV.
NOTE: The data set WORK.COV has 150 observations and 15 variables.
NOTE: DATA statement used (Total process time):
real time 0.01 seconds
cpu time 0.00 seconds

NOTE: There were 150 observations read from the data set WORK.COV.
WHERE col1 is not null;
NOTE: The data set WORK.COV has 150 observations and 15 variables.
NOTE: DATA statement used (Total process time):
real time 0.00 seconds
cpu time 0.00 seconds

67
68 data cov_imput;
69 merge nlcovb(keep=_imputation_ row parameter) cov;
70 run;

NOTE: There were 150 observations read from the data set WORK.NLCOVB.
10 The SAS System 09:59 Monday, April 22, 2024

NOTE: There were 150 observations read from the data set WORK.COV.
NOTE: The data set WORK.COV_IMPUT has 150 observations and 18 variables.
NOTE: DATA statement used (Total process time):
real time 0.01 seconds
cpu time 0.00 seconds

71 data cov_imput;
72 set cov_imput;
73 rename
74 COL1=int1
75 col2=beta_ns1
76 col3=beta_cm1
77 col4=beta_b1
78 col5=int2
79 col6=beta_ns2
80 col7=beta_cm2
81 col8=beta_b2
82 col9=int3
83 col10=beta_ns3
84 col11=beta_cm3
85 col12=beta_b3
86 col13=r12
87 col14=r23
88 col15=r13;
89 run;

NOTE: There were 150 observations read from the data set WORK.COV_IMPUT.
NOTE: The data set WORK.COV_IMPUT has 150 observations and 18 variables.
NOTE: DATA statement used (Total process time):
real time 0.00 seconds
cpu time 0.00 seconds

90
91 proc mianalyze parms=mvp.parms_imput covb=cov_imput wcov bcov tcov;
92 modeleffects
93 int1
94 beta_ns1
95 beta_cm1
96 beta_b1
97 int2
98 beta_ns2
99 beta_cm2
100 beta_b2
101 int3
102 beta_ns3
103 beta_cm3
104 beta_b3
105 r12
106 r13
107 r23;
108 ods output ParameterEstimates=mvp.result_imput;
109 run;

ERROR: Within-imputation COVB matrix is not symmetric for _Imputation_= 1 in the input COVB= data set.
NOTE: The SAS System stopped processing this step because of errors.
NOTE: PROCEDURE MIANALYZE used (Total process time):
11 The SAS System 09:59 Monday, April 22, 2024

real time 0.00 seconds
cpu time 0.00 seconds

WARNING: Output 'ParameterEstimates' was not created. Make sure that the output object name, label, or path is spelled correctly.
Also, verify that the appropriate procedure options are used to produce the requested output object. For example, verify
that the NOPRINT option is not used.

Ksharp · Posted 04-22-2024 09:47 PM

Why not post it at IML forum ?
https://communities.sas.com/t5/SAS-IML-Software-and-Matrix/bd-p/sas_iml

Rounding issues

Re: Rounding issues

Re: Rounding issues

Re: Rounding issues

Rounding issues

Re: Rounding issues

Re: Rounding issues

Re: Rounding issues

Register Today!

SAS Training: Just a Click Away