topic Re: Compare two models in Statistical Procedures

Compare two models

Raza_M — Fri, 17 Jun 2022 08:56:31 GMT

Hi everyone,

I used PROC GENMOD using GEE for repeated measures to find an association between dichotomized outcome and continuous exposure. Then to see difference by sex, I used interaction term and reported sex stratified results if the interaction term was significant. However, reviewer is not satisfied with this method and wants some kind of comparison between the models and test if the difference is significant.

I browsed the internet and is seems that using a CONTRAST statement would have solved the issue but unfortunately it cannot be used with repeated measures (GEE). However, if I use CONSTRACT in PROC GENMOD with GEE it produces a test score for hypothesis. but I am not sure if it is the same thing.

Can you kindly guide how to do this correctly? I have written below a code for the main model and the interaction model which I used separately and now I am looking for a comparison between these two models to test if the models are different.

Main model:

proc genmod data=dateused descending;
class sex id v1(ref='0') v2(ref='0') v3(ref='1') v4(ref='1') v5(ref='0') ;
model outcome_dichotomized = exposure_continous sex age v1 v2 v3 v4 v5 / dist=bin link=logit type3;
estimate 'exposure_continous ' exposure_continous -1 /exp;
repeated subject=id/ type=exch;
run;

Interaction model:

proc genmod data=dateused descending;
class sex id v1(ref='0') v2(ref='0') v3(ref='1') v4(ref='1') v5(ref='0') ; ;
model outcome_dichotomized = exposure_continous sex age v1 v2 v3 v4 v5 exposure_continous *sex / dist=bin link=logit type3;
estimate 'exposure_continous ' exposure_continous -1 /exp;
repeated subject=id/ type=exch;
run;

Re: Compare two models

sbxkoenk — Fri, 17 Jun 2022 13:50:04 GMT

Have moved this from 'Programming' to 'Statistical Procedures' board.

Koen

Re: Compare two models

sbxkoenk — Fri, 17 Jun 2022 13:52:30 GMT

Sample 26100: QIC goodness of fit statistic for GEE models
https://support.sas.com/kb/26/100.html

These statistics allow comparisons of GEE models (model selection) and selection of a correlation structure.

Koen

Re: Compare two models

StatDave — Fri, 17 Jun 2022 14:37:30 GMT

The question of whether your exposure variable has difference effects in each sex is directly answered by the test of the interaction parameter in your model that includes the interaction. If the test of that parameter (or its type 3 test) is significant, then exposure has different effects in the two sexes.

Re: Compare two models

Raza_M — Fri, 17 Jun 2022 16:24:34 GMT

Yes, I know but reviewer does not accept this and want us to run some kind of test to compare the models to show that the main model is different from the model with interaction term

Re: Compare two models

Raza_M — Fri, 17 Jun 2022 16:28:24 GMT

But QIC does not tell if two models are different and interaction is significant

Re: Compare two models

Raza_M — Fri, 17 Jun 2022 16:28:52 GMT

Ok, Thank you!

Re: Compare two models

Raza_M — Fri, 17 Jun 2022 16:34:36 GMT

Reviewer wants something like I run two models: One with all main effects and one with the main effects and interaction terms. If the explanatory power of the interaction model is significantly higher, I interpret the interaction. But I do not know how to achieve this in SAS

Re: Compare two models

StatDave — Fri, 17 Jun 2022 22:55:08 GMT

Your reviewer is thinking of a likelihood ratio test which is computed that way by fitting the separate models and comparing their likelihood values. And that is an excellent test to use, but it requires that the models be fit using maximum likelihood estimation. Unfortunately, the GEE method is not a likelihood-based method, so that test is not possible. The parameter test is directly analogous to what you would do for an ordinary regression model to assess this question and should be perfectly acceptable.

Re: Compare two models

sbxkoenk — Sat, 18 Jun 2022 09:38:24 GMT

Hello,

On top of what @StatDave has written :

You might have a look into this (1993 Sep 30) article by T Park :

A comparison of the generalized estimating equation approach with the maximum likelihood approach for repeated measurements

https://pubmed.ncbi.nlm.nih.gov/8248664/

I think it's safe to say GEE is QMLE (Quasi-Maximum-Likelihood-Estimation), but it is definitely not MLE !

Koen