BookmarkSubscribeRSS Feed
🔒 This topic is solved and locked. Need further help from the community? Please sign in and ask a new question.
suzan
Fluorite | Level 6

Dear All,

I am using SAS 9.4.

I have fit three,  2-level model to my data set using Proc glimmix. as follows:

The outcome(Diabetes) and  the covariate "bp" are binary variables.

Fitted models are :

1- full model with random intercept (A)

2- Full model with both random intercept and random slope(A1)

3- Model A2: excluding the intercation term from fixed level 1 covariates

I just wonder how can I compare these three models usyng SAS and  tell which model fits better and are more significant

 

I appreciate it very much for your help,

Smiley Happy

 

 

 


ods output fitstatistics=fitA; title "Model A,random intercept+fixed interactions"; proc glimmix data=data1 noclprint; class group bp ; model DIABETES = bmi bp bmi*bp/ solution link=logit dist=binary ddfm=satterthwaite ; random intercept / type=un subject=group; weight WEIGHT; format bp bp.; format DIABETES DIABETES.; run;


ods output fitstatistics=fitA1;
title "Model A,random intercept+fixed interactions";
proc glimmix data=data1 noclprint;
class group bp ;
model DIABETES = bmi bp bmi*bp/ solution link=logit dist=binary ddfm=satterthwaite ;
random intercept bmi/ type=un subject=group;
weight WEIGHT;
covtest "random slope" . 0 0;
format bp bp.;
format DIABETES DIABETES.;
run;

ods output fitstatistics=fitA2;
title "Model A,random intercept+fixed interactions";
proc glimmix data=data1 noclprint;
class group bp ;
model DIABETES = bmi bp / solution link=logit dist=binary ddfm=satterthwaite ;
random intercept / type=un subject=group;
weight WEIGHT;
format bp bp.;
format DIABETES DIABETES.;
run;

 

1 ACCEPTED SOLUTION

Accepted Solutions
lvm
Rhodochrosite | Level 12 lvm
Rhodochrosite | Level 12

You can compare AIC or AICC values for models that differ only in the random effects, when using the default REML estimation. You cannot do this for models that differ in the fixed effects. If you want to use likelihood-based comparison methods, use method=mspl in the GLIMMIX statement, which will get you ML estimation. However, ML estimation can give biased estimates of variances, which affects test statistics, etc. The bias could be small with a very large data set, but large with a small data set.

 

Editor's note: see also the clarifications by @StatsMan in a later reply:

IVM is correct if the modeling is done in MIXED or if you are using GLIMMIX with normal errors.  If you have binary data, though, the default estimation uses pseudo-likelihood methods so direct comparisons of the likelihoods and likelihood based statistics between competing models is not advisable.

 

If you switch to METHOD=LAPLACE or METHOD=QUAD, then a direct comparison can be made.

 

See the Fit Statistics section of the PROC GLIMMIX documentation for details.



 

View solution in original post

5 REPLIES 5
lvm
Rhodochrosite | Level 12 lvm
Rhodochrosite | Level 12

You can compare AIC or AICC values for models that differ only in the random effects, when using the default REML estimation. You cannot do this for models that differ in the fixed effects. If you want to use likelihood-based comparison methods, use method=mspl in the GLIMMIX statement, which will get you ML estimation. However, ML estimation can give biased estimates of variances, which affects test statistics, etc. The bias could be small with a very large data set, but large with a small data set.

 

Editor's note: see also the clarifications by @StatsMan in a later reply:

IVM is correct if the modeling is done in MIXED or if you are using GLIMMIX with normal errors.  If you have binary data, though, the default estimation uses pseudo-likelihood methods so direct comparisons of the likelihoods and likelihood based statistics between competing models is not advisable.

 

If you switch to METHOD=LAPLACE or METHOD=QUAD, then a direct comparison can be made.

 

See the Fit Statistics section of the PROC GLIMMIX documentation for details.



 

suzan
Fluorite | Level 6

Thank you  very much. It was really helpful.

Have tried  your solution.  I just wonder after adding this option into my each an every model what should I expect to see differently in the output.(I noticed that for each model, fit statistics [-2 log likelihood] is changed after adding the mspl option) .

My undersanding is that I have  to compare this part of analysis for all models.Sorry for such a question! I am a begginer !Smiley Happy

 

StatsMan
SAS Super FREQ

IVM is correct if the modeling is done in MIXED or if you are using GLIMMIX with normal errors.  If you have binary data, though, the default estimation uses pseudo-likelihood methods so direct comparisons of the likelihoods and likelihood based statistics between competing models is not advisable.

 

If you switch to METHOD=LAPLACE or METHOD=QUAD, then a direct comparison can be made.

 

See the Fit Statistics section of the PROC GLIMMIX documentation for details.

lvm
Rhodochrosite | Level 12 lvm
Rhodochrosite | Level 12

Yes, it is important to use method=laplace or method=quad for binary or binomial data. (I missed that you were dealing with binary). Both laplace and quad are maximum likelihood methods, so the principles I mentioned still hol. Use method=mspl for this purpose for normal distributions if you want to compare likelihoods (or the AIC and AICC statistics).

A general rule is the smaller the AIC, the better the fit. You want a change in AIC of at least 2 to consider this a better fit.

suzan
Fluorite | Level 6

Many thanks for your guidance. It helped big timeSmiley Happy

sas-innovate-2024.png

Join us for SAS Innovate April 16-19 at the Aria in Las Vegas. Bring the team and save big with our group pricing for a limited time only.

Pre-conference courses and tutorials are filling up fast and are always a sellout. Register today to reserve your seat.

 

Register now!

What is ANOVA?

ANOVA, or Analysis Of Variance, is used to compare the averages or means of two or more populations to better understand how they differ. Watch this tutorial for more.

Find more tutorials on the SAS Users YouTube channel.

Discussion stats
  • 5 replies
  • 5000 views
  • 5 likes
  • 3 in conversation