topic Re: likelihood ratio test in mixed model in Statistical Procedures

likelihood ratio test in mixed model

GiaLee — Wed, 20 Mar 2024 02:01:26 GMT

I would like to compare two nested linear mixed models: one without interaction (referred to as model 1) and one with interaction (referred to as model 2).

Interaction term: time (6 classes) x gender (2 classes).

My mentor instructed me to compare these models using the Likelihood Ratio Test. I have reviewed many instructions but still feel confused about a few aspects.

1. For the difference in DF, adding the gender*time interaction would increase the parameters by (6-1) * (2-1)=5. Am I understand it correctly?

2. I conducted tests for both models. Model 1: -2LL=3267, and for Model 2, -2LL=3272. This indicates that adding the interaction increased the deviance and decreased the model fit. In this case, should I proceed with the Likelihood Ratio Test? Because all the instructional materials I have reviewed so far assume that adding variables decreases the deviance. So they calculate the -2LL (reduced model) - -2LL( full model), and if the p is significant, the full model works better. In my case, it is reverse.

If I can proceed with the test, the deviance difference would be 3272-3267=5. Assuming a DF difference of 5, test statistic would be 11.07, and a p-value of 0.041. Does this imply that Model 1 (without the interaction term) improve the results? It sounds a little bit strange..

Thanks for any answers on these questions.

Re: likelihood ratio test in mixed model

Ksharp — Wed, 20 Mar 2024 02:55:39 GMT

Here is an example for PROC GENMOD.
But I think the same rule could apply to PROC MIXED .

24447 - Examples of writing CONTRAST and ESTIMATE statements (sas.com)

Re: likelihood ratio test in mixed model

GiaLee — Wed, 20 Mar 2024 03:22:21 GMT

Thanks for your quick reply. This notes answered my first question, thanks!

However, the example is still like all other instructions show: adding an interaction term decreases the deviance. And then, they did the LRT.
While in my case, adding an interaction term increases the deviance. And I don't know if I can still perform the LRT in this situation.

Re: likelihood ratio test in mixed model

StatsMan — Wed, 20 Mar 2024 13:37:05 GMT

Posting the code used and actual output will help the community answer your question more quickly and more accurately. Comparison of likelihood stats from competing mixed models can by tricky. Changes in the model can make the comparisons can make those comparisons invalid. Seeing the code will help the community provide a better response. Thanks!

Re: likelihood ratio test in mixed model

GiaLee — Wed, 20 Mar 2024 14:22:51 GMT

Thanks for your reply. Here is the code. I compared the reduced model with the full model. The only difference is the interaction term. I found that adding interaction terms increased the -2LL (3271 - 3286). So I wonder if this result is correct, and if I could still do the Likelihood ratio test. If yes, will the hypothesis be changed to whether excluding the interaction term helps with model fit?

Reduced model:

PROC MIXED DATA = test METHOD = REML COVTEST;
CLASS record_id time_(ref="6") female(ref="0") race_JL (ref="White")
depression_(ref="0") anxiety_(ref="0") hypertension_(ref="0") overweight(ref="0") fatigue_JL(ref="0")
site_JL(ref="outpatien") vaccine(ref="0") comor_cat (ref="none or mild");
MODEL Score = time_ female race_JL depression_ anxiety_
hypertension_ overweight fatigue_JL site_JL vaccine comor_cat/ SOLUTION OUTpred=PREDDATA S CL;
RANDOM INTERCEPT / SUBJECT = record_id;
repeated time_ / type=ar(1) sub=record_id; store out=MixedModel;
RUN;

Full model:
PROC MIXED DATA = all_long_1 METHOD = REML COVTEST;
CLASS record_id time_(ref="6") female(ref="0") race_JL (ref="White")
depression_(ref="0") anxiety_(ref="0") hypertension_(ref="0") overweight(ref="0") fatigue_JL(ref="0")
site_JL(ref="outpatien") vaccine(ref="0") comor_cat (ref="none or mild");
MODEL Score = time_ female time_*female race_JL depression_ anxiety_
hypertension_ overweight fatigue_JL site_JL vaccine comor_cat/ SOLUTION OUTpred=PREDDATA S CL;
RANDOM INTERCEPT / SUBJECT = record_id;
repeated time_ / type=ar(1) sub=record_id; store out=MixedModel;
RUN;

Re: likelihood ratio test in mixed model

GiaLee — Wed, 20 Mar 2024 15:04:05 GMT

I may have found the reason for the problem:

"the problem was coming from mixed model using restricted maximum likelihood (REML) instead of maximum likelihood (ML) for the parameters optimization. This makes the likelihood not comparable between two models having different fixed effects”.

"The reason is that REML estimates the random effects by considering linear combinations of the data that remove the fixed effects. If these fixed effects are changed, the likelihoods of the two models will not be directly comparable”.

So does this mean the mixed model cannot perform the LRT test since it only uses REML?

(I tried changing the syntax from "method = reml" to "method = ml", which gave us the -2LL instead of -2resLL, but it also changed the parameter estimates)

Re: likelihood ratio test in mixed model

StatsMan — Wed, 20 Mar 2024 16:35:57 GMT

You can switch to METHOD=ML and compare the likelihoods if you use the same covariance structures in both models. The parameter estimates will change if you change the fixed effects. That is not unexpected.

Re: likelihood ratio test in mixed model

GiaLee — Wed, 20 Mar 2024 16:47:57 GMT

Thanks! It works!