topic Re: Huge difference between Likelihood ratio test and t value in proc glimmix in Statistical Procedures

Huge difference between Likelihood ratio test and t value in proc glimmix

boban — Tue, 13 Nov 2018 16:19:05 GMT

Hi,

I am using a log-likelihood ratio test and the t value, in proc glimmix, to compare the fit of two nested models – one random intercept model and one full model, by adding a continuous explanatory variable to the model.

I obtained hugely contrasting results between the LL ratio test and the reported t value!

While the LL ratio test was highly significant, the t value was highly non-significant.

Does anyone know what may be causing this, and how I should interpret the direction of the trend for the logistic regression (i.e. positive or negative effect of the predictor variable on the probability of the event occurring)?

Thank you in advance,

Boban

Re: Huge difference between Likelihood ratio test and t value in proc glimmix

PaigeMiller — Tue, 13 Nov 2018 16:23:15 GMT

Are these testing the same hypotheses in the same model? Or are they from different models? Can you show us the relevant parts of the SAS code? Can you show us the relevant parts of the SAS output?

Re: Huge difference between Likelihood ratio test and t value in proc glimmix

boban — Wed, 14 Nov 2018 04:25:25 GMT

Thank you for your reply!

I am testing only one hypothesis, the effect of jsh on the outcome surv, shown in the full model bellow:

proc glimmix method = laplace data=finale_updated;

class year;

model surv (event = '1')= jsh /S dist = binary link = logit ddfm =bw;

random int/ subject = year;

covtest 'var (year)= 0' 0 / est parms ;

run;

Here is the output for the full model:

Solutions for Fixed EffectsEffect Estimate StandardError DF t Value Pr > |t|InterceptJSH

-2.7900	0.5244	8	-5.32	0.0007
-0.6452	0.4951	2789	-1.30	0.1926

Fit Statistics-2 Log LikelihoodAIC (smaller is better)AICC (smaller is better)BIC (smaller is better)CAIC (smaller is better)HQIC (smaller is better)

846.75

852.75

852.76

853.34

856.34

851.48

And for the empty model, without the variable jsh

Fit Statistics-2 Log LikelihoodAIC (smaller is better)AICC (smaller is better)BIC (smaller is better)CAIC (smaller is better)HQIC (smaller is better)

2170.83

2174.83

2174.84

2175.23

2177.23

2173.98

By comparing the two outputs, you see that the p value for t is very high. However, the difference in -2 Log Likelihood is just as important, and all information criteria!

I am wondering what may be causing this?

What shall I conclude?

How do I interpret the odds ratio, not shown here, but there is a negative relationship.

Best,

Boban

Re: Huge difference between Likelihood ratio test and t value in proc glimmix

boban — Wed, 14 Nov 2018 04:29:32 GMT

Sorry, but it looks as if the formating ot the tables didn't work.

Anyway, the p value for t is 0.1926, shown in the first table.

In the other two tables, the -2 log likelihood is the value at the top.

Boban

Re: Huge difference between Likelihood ratio test and t value in proc glimmix

PaigeMiller — Wed, 14 Nov 2018 14:35:42 GMT

It seems like you are comparing a t-test about whether or not a term in the model is significant, to a model fit statistic. Is that what you are doing? These are not the same and don't have to be close to one another.

Re: Huge difference between Likelihood ratio test and t value in proc glimmix

sld — Wed, 14 Nov 2018 18:55:25 GMT

Did you include the RANDOM statement in the reduced model?

Re: Huge difference between Likelihood ratio test and t value in proc glimmix

boban — Thu, 15 Nov 2018 05:22:53 GMT

I am testing the significance of a fixed effect, using both the t statitic and the LL ratio test (model fit estimation). They are not always supposed to yield simmilar results, but the difference is very important here!

I am wondering how to interpret this. Thank you for spending time on this!

Yes, the random effect is included in both.

Re: Huge difference between Likelihood ratio test and t value in proc glimmix

PaigeMiller — Thu, 15 Nov 2018 12:59:50 GMT

@boban wrote:

I am testing the significance of a fixed effect, using both the t statitic and the LL ratio test (model fit estimation). They are not always supposed to yield simmilar results, but the difference is very important here!

I am wondering how to interpret this. Thank you for spending time on this!

These are not the same test. They are not testing the same hypothesis. They are not on the same scale. There is no reason to expect similar results. They are interpreted as individual tests, not related to one another.

Re: Huge difference between Likelihood ratio test and t value in proc glimmix

sld — Thu, 15 Nov 2018 16:24:00 GMT

Hmm. To have any more (potential) insight, I would need to see the code for the reduced model, and ideally the full output for both full and reduced models (attached as a file, e.g., PDF, not embedded in your message; as you've noticed, the tables don't embed well).