How do I compare mixed models with proc glimmix

[suzan]

Dear All,

I am using SAS 9.4.

I have fit three,  2-level model to my data set using Proc glimmix. as follows:

The outcome(Diabetes) and  the covariate "bp" are binary variables.

Fitted models are :

1- full model with random intercept (A)

2- Full model with both random intercept and random slope(A1)

3- Model A2: excluding the intercation term from fixed level 1 covariates

I just wonder how can I compare these three models usyng SAS and  tell which model fits better and are more significant

I appreciate it very much for your help,

ods output fitstatistics=fitA;
title "Model A,random intercept+fixed interactions";
proc glimmix data=data1 noclprint;
class group bp ;
model DIABETES = bmi bp bmi*bp/ solution link=logit dist=binary ddfm=satterthwaite ;
random intercept / type=un subject=group;
weight WEIGHT;
format bp bp.;
format DIABETES DIABETES.;
run;


ods output fitstatistics=fitA1;
title "Model A,random intercept+fixed interactions";
proc glimmix data=data1 noclprint;
class group bp ;
model DIABETES = bmi bp bmi*bp/ solution link=logit dist=binary ddfm=satterthwaite ;
random intercept bmi/ type=un subject=group;
weight WEIGHT;
covtest "random slope" . 0 0;
format bp bp.;
format DIABETES DIABETES.;
run;

ods output fitstatistics=fitA2;
title "Model A,random intercept+fixed interactions";
proc glimmix data=data1 noclprint;
class group bp ;
model DIABETES = bmi bp / solution link=logit dist=binary ddfm=satterthwaite ;
random intercept / type=un subject=group;
weight WEIGHT;
format bp bp.;
format DIABETES DIABETES.;
run;

[lvm · Accepted solution]

You can compare AIC or AICC values for models that differ only in the random effects, when using the default REML estimation. You cannot do this for models that differ in the fixed effects. If you want to use likelihood-based comparison methods, use method=mspl in the GLIMMIX statement, which will get you ML estimation. However, ML estimation can give biased estimates of variances, which affects test statistics, etc. The bias could be small with a very large data set, but large with a small data set.

Editor's note: see also the clarifications by @StatsMan in a later reply:

IVM is correct if the modeling is done in MIXED or if you are using GLIMMIX with normal errors.  If you have binary data, though, the default estimation uses pseudo-likelihood methods so direct comparisons of the likelihoods and likelihood based statistics between competing models is not advisable.

If you switch to METHOD=LAPLACE or METHOD=QUAD, then a direct comparison can be made.

See the Fit Statistics section of the PROC GLIMMIX documentation for details.

---

[suzan]

Thank you  very much. It was really helpful.

Have tried  your solution.  I just wonder after adding this option into my each an every model what should I expect to see differently in the output.(I noticed that for each model, fit statistics [-2 log likelihood] is changed after adding the mspl option) .

My undersanding is that I have  to compare this part of analysis for all models.Sorry for such a question! I am a begginer !

[StatsMan]

IVM is correct if the modeling is done in MIXED or if you are using GLIMMIX with normal errors.  If you have binary data, though, the default estimation uses pseudo-likelihood methods so direct comparisons of the likelihoods and likelihood based statistics between competing models is not advisable.

If you switch to METHOD=LAPLACE or METHOD=QUAD, then a direct comparison can be made.

See the Fit Statistics section of the PROC GLIMMIX documentation for details.

[lvm]

Yes, it is important to use method=laplace or method=quad for binary or binomial data. (I missed that you were dealing with binary). Both laplace and quad are maximum likelihood methods, so the principles I mentioned still hol. Use method=mspl for this purpose for normal distributions if you want to compare likelihoods (or the AIC and AICC statistics).

A general rule is the smaller the AIC, the better the fit. You want a change in AIC of at least 2 to consider this a better fit.

[suzan]

Many thanks for your guidance. It helped big time