Programming the statistical procedures from SAS

Why does changing covariance structure radically change the standard error of the fixed effects?

Accepted Solution Solved
Reply
Frequent Contributor
Posts: 147
Accepted Solution

Why does changing covariance structure radically change the standard error of the fixed effects?

[ Edited ]

Using SAS 9.4.

 

16 animals were randomly assigned to two treatments (control and treatment). Each animal had its level on the dependent variable measured 24 times over the course of the day. For each animal, these numbers vary wildly, from 0 into the thousands, with no strong correlations.

 

Interest is in whether the two treatments are different.

 

I compared three models:

a) T-test on total activity level

proc ttest data = wide;
 class group;
 var total;
run;

 

b) A multilevel model with unstructured covariances

title "MLM for day 1, unstructured covariance, only group entered";
proc mixed data = long;
 class group mouse_id;
 model act = group/solution;
 repeated/type = un subject = mouse_id;
run;

c) A multilevel model with AR(1) covariances

 

title "MLM for day 1, AR1 covariance, only group entered";
proc mixed data = long;
 class group mouse_id;
 model act = group/solution;
 repeated/type = ar(1) subject = mouse_id;
run;

c) Had slightly better AIC and BIC than b).

 

The effect size of group was roughly equal across all three models (after accounting for the fact that the t-test was on 24 values added up) but the standard errors for b) and c) were wildly different. b) Was significant, a) and c) were not.

 

In the past, I've not seen such huge changes in the standard errors.

 

Any thoughts on why this might occur and what to do?

 

NOTE: It's the same data, just one in long format and one in wide format.


Accepted Solutions
Solution
‎04-12-2018 07:50 AM
Respected Advisor
Posts: 3,018

Re: Why does changing covariance structure radically change the standard error of the fixed effects?

There's not much we can say since we don't have your data, and you have changed the data sets from one analysis to another anyway (not even sure how you can compare different data sets in this way).

 

But ... if you fit a model with the wrong covariance structure (which doesn't match the data), I can understand why the model doesn't fit well and the standard errors are large.

--
Paige Miller

View solution in original post


All Replies
Solution
‎04-12-2018 07:50 AM
Respected Advisor
Posts: 3,018

Re: Why does changing covariance structure radically change the standard error of the fixed effects?

There's not much we can say since we don't have your data, and you have changed the data sets from one analysis to another anyway (not even sure how you can compare different data sets in this way).

 

But ... if you fit a model with the wrong covariance structure (which doesn't match the data), I can understand why the model doesn't fit well and the standard errors are large.

--
Paige Miller
☑ This topic is solved.

Need further help from the community? Please ask a new question.

Discussion stats
  • 1 reply
  • 95 views
  • 1 like
  • 2 in conversation