Wrong confidence limits for predicted values in simple linear regression with Proc Mixed

[WillTheKiwi]

I've been using outp and outpm in the model statement in Proc Mixed. The outp gives the usual prediction interval for individual observations when there is a random or repeated statement, but when it's a simple linear regression without such statements, the outp confidence limits are the same as the outpm confidence limits.  I tried adding a dummy repeated (repeated;), but it still gave the confidence interval for the predicted mean.  Proc Reg gave the right intervals--well, I presume they are right; they were wider than those for the mean! I'll provide data and code if I have to, but trust me, it's simple enough and it simply isn't right. It's the latest version of SAS Studio University Edition. It's actually a PhD student's project. He's using Proc Reg for the simple linear regressions meantime, but it's a hassle, because he has lots of models and dependent variables. Anyway, it needs to be fixed.

Will

[ballardw]

Without, at an absolute minimum, code of the two approaches there isn't much anyone here can do to "fix" anything.

And the reasons for using Proc Mixed would make me suspect that any result from Proc Reg should be different. That is why there are multiple regression procedures. Some will generate similar output for the same data but since other options such as Repeated or Random are not in Proc Reg ...

[WillTheKiwi]

The student has provided some code to produce and compare the predicted values with Proc Mixed and Proc Reg in a simple linear regression. Proc Mixed produces the same confidence limits for the predicted mean values as Proc Reg, but Proc Mixed produces the same confidence limits for the predicted individual values, whereas Proc Reg produces different, wider confidence limits.

Whatever you may think, I regard this as a substantial problem. I have been doing random-effect meta-analyses with Proc Mixed and using predicted values in meta-regressions to get predicted meta-analyzed effects in mean settings and in individual settings (for given values of the study characteristics--I create dummy study-estimates with missing values of the dependent variable to achieve this, elegantly).  I no longer trust the predicted confidence limits defining the prediction interval for individual settings.

Please get this problem fixed.

Will

[sld · Accepted solution]

In MIXED, the outp option produces predicted values as Xb + Zu, i.e., EBLUPs. The outpm option produces predicted values as just Xb, i.e., EBLUEs. In both cases, intervals are confidence intervals, not prediction intervals. Here is the documentation link.

In your example, the outp and outpm predictions from MIXED are the same because the model does not have a Zu component.

I don't think that MIXED has the capability to produce prediction intervals directly. I thought that I might be able to acquire prediction intervals using PROC PLM, but I wasn't successful with my small test which reported the "prediction limits are not available in this model". But you could check with SAS Tech Support. (I'll note that if, in fact, this issue was an error, you should report it directly to Tech Support, not the SAS Community. Most of us in the Community do not work for SAS. But I think the issue is a misunderstanding, not an error.)

I hope this helps.

[WillTheKiwi]

Thanks for the reasoned and quick reply. If I understand you correctly,
Proc Mixed does not include anything from the residuals in its outp
confidence limits. Maybe that should be an option to make simple regression
models concur exactly with those estimated with Proc Reg. Anyway, if that
is the case, the prediction intervals from my meta-regressions should be OK,
because I use the clever code that Yang (2003) devised, in which you set the
residual variance to 1. I have several random effects, and I get wider
intervals for the outp than for the outpm. Presumably these are trustworthy
prediction intervals in individual settings and the predicted mean
confidence limits for mean of such settings, respectively.



Yang M. A Review of Random Effects Modelling in SAS (Release 8.2). London:
London, Centre for Multilevel Modelling, 2003.

[sld]

I think your statement about inclusion of residuals is true.

I would also think that computation of prediction intervals in a mixed model might be complex. If I ever have that problem, I'll check out the Yang reference. Thanks.

[WillTheKiwi]

I have sent a request to SAS Technical Support.  I also have some additional information. The PhD student informs me that when he has a repeated statement that specifies more than one residual, and no random statement, the outp confidence limits are now different from those of outpm. They seem to be believable as prediction intervals. So Proc Mixed with a dummy or no repeated statement and no random statement should produce the same prediction intervals as Proc Reg, but obviously it doesn't.

I guess I can stop worrying about the trustworthiness of the prediction intervals in my random-effect meta-regressions, when I use the Yang trick of setting the residual variance to unity. I guess I should check by using an estimate statement that includes random effects to reproduce a predicted value. The confidence intervals should be the same, hopefully correct!