topic Re: Lack of fit test in Proc Mixed? in Statistical Procedures

Lack of fit test in Proc Mixed?

LauraPerry — Thu, 29 May 2014 20:07:01 GMT

What is the best method for testing for lack of fit in models fit using Proc Mixed? I'm running models testing effects of river reach, forest stand age, elevation above the river, and distance from the river on soils in riparian forests. Samples were collected from plots located within distinct forest patches, arrayed along randomly located transects within each river reach, so I include patch nested within transect and transect nested within reach as random effects. I've pasted the code below. I'm running these models to assess the relative importance of stand age, elevation, and distance by comparing AICc scores among models that include different combinations of the variables. I'd like to have a method to check whether the "best model" (i.e., the model with the lowest AICc) provides a good fit to the data. Thanks in advance for any advice.

- Laura

title soilN - age elev dist;

proc mixed;

class reach transect patch;

model N=age elev dist reach / solution ddfm=kenwardroger;

random transect(reach) patch(transect*reach);

run;

Re: Lack of fit test in Proc Mixed?

SteveDenham — Fri, 30 May 2014 14:37:31 GMT

You can look at the change in the AIC or AICc between models. For example, check out http://en.wikipedia.org/wiki/Akaike_information_criterion and http://en.wikipedia.org/wiki/Likelihood_function#Relative_likelihood where the formula exp((AIC - AICmin)/2) [or replace AIC with AICc] gives the relative information loss. In this case, look at the comparison of your model to the null model. This will tell you how much of the information in the data is retained in your model. I'm pretty sure a search through the Statistics forum will bring up another thread that covered this as well.

Steve Denham

Re: Lack of fit test in Proc Mixed?

LauraPerry — Fri, 30 May 2014 15:32:58 GMT

Thank you - this is helpful. A question: what would be the null model in this case? I don't know how to write proc mixed for a model with no independent variables in it.

Re: Lack of fit test in Proc Mixed?

SteveDenham — Fri, 30 May 2014 15:44:26 GMT

Since the random effects are design elements, I would leave them in and run:

proc mixed;

class reach transect patch;

model N= / solution ddfm=kenwardroger;

random transect(reach) patch(transect*reach);

run;

Does this make sense?

Steve Denham

Re: Lack of fit test in Proc Mixed?

lvm — Fri, 30 May 2014 15:45:17 GMT

Be careful. The default estimation method in MIXED is REML. That means, you can only compare AIC statistics for models with different random effects and the same fixed effects (the part in the model statement). If you want to compare models with different fixed effects, you will have to use method=ml in the procedure statement. Results could be a bit biased with small sample sizes.

The null model is easy, nothing before the slash.

model N = / solution ddfm = kr;

Re: Lack of fit test in Proc Mixed?

SteveDenham — Fri, 30 May 2014 17:08:58 GMT

Thanks, Larry. I forgot (AGAIN) that it had to be method=ML in PROC MIXED, and either method=laplace or method=quad in PROC GLIMMIX to look at fixed effect contributions to information criteria.

Steve Denham

Re: Lack of fit test in Proc Mixed?

LauraPerry — Fri, 30 May 2014 17:28:37 GMT

Thanks Steve and Larry. The null model comparison seems to work well. And you rescued me from a serious blunder with the REML/ML method. Thanks.