Programming the statistical procedures from SAS

Lack of fit test in Proc Mixed?

Reply
New Contributor
Posts: 3

Lack of fit test in Proc Mixed?

What is the best method for testing for lack of fit in models fit using Proc Mixed?  I'm running models testing effects of river reach, forest stand age, elevation above the river, and distance from the river on soils in riparian forests.  Samples were collected from plots located within distinct forest patches, arrayed along randomly located transects within each river reach, so I include patch nested within transect and transect nested within reach as random effects.  I've pasted the code below.  I'm running these models to assess the relative importance of stand age, elevation, and distance by comparing AICc scores among models that include different combinations of the variables.  I'd like to have a method to check whether the "best model" (i.e., the model with the lowest AICc) provides a good fit to the data.  Thanks in advance for any advice.

- Laura

title soilN - age elev dist;

proc mixed;

class reach transect patch;

model N=age elev dist reach / solution ddfm=kenwardroger;

random transect(reach) patch(transect*reach);

run;

Respected Advisor
Posts: 2,655

Re: Lack of fit test in Proc Mixed?

You can look at the change in the AIC or AICc between models.  For example, check out http://en.wikipedia.org/wiki/Akaike_information_criterion  and http://en.wikipedia.org/wiki/Likelihood_function#Relative_likelihood where the formula exp((AIC - AICmin)/2) [or replace AIC with AICc] gives the relative information loss.  In this case, look at the comparison of your model to the null model.  This will tell you how much of the information in the data is retained in your model.  I'm pretty sure a search through the Statistics forum will bring up another thread that covered this as well.

Steve Denham

New Contributor
Posts: 3

Re: Lack of fit test in Proc Mixed?

Thank you - this is helpful.  A question: what would be the null model in this case?  I don't know how to write proc mixed for a model with no independent variables in it.

Respected Advisor
Posts: 2,655

Re: Lack of fit test in Proc Mixed?

Since the random effects are design elements, I would leave them in and run:

proc mixed;

class reach transect patch;

model N= / solution ddfm=kenwardroger;

random transect(reach) patch(transect*reach);

run;

Does this make sense?

Steve Denham

Valued Guide
Valued Guide
Posts: 673

Re: Lack of fit test in Proc Mixed?

Be careful. The default estimation method in MIXED is REML. That means, you can only compare AIC statistics for models with different random effects and the same fixed effects (the part in the model statement). If you want to compare models with different fixed effects, you will have to use method=ml in the procedure statement. Results could be a bit biased with small sample sizes.

The null model is easy, nothing before the slash.

model N = / solution ddfm = kr;

Respected Advisor
Posts: 2,655

Re: Lack of fit test in Proc Mixed?

Thanks, Larry.  I forgot (AGAIN) that it had to be method=ML in PROC MIXED, and either method=laplace or method=quad in PROC GLIMMIX to look at fixed effect contributions to information criteria.

Steve Denham

New Contributor
Posts: 3

Re: Lack of fit test in Proc Mixed?

Thanks Steve and Larry.  The null model comparison seems to work well.  And you rescued me from a serious blunder with the REML/ML method.  Thanks.

Ask a Question
Discussion stats
  • 6 replies
  • 394 views
  • 1 like
  • 3 in conversation