topic Re: Do I need to stick to mixed effect model when random effect is insignficant? in Statistical Procedures

Do I need to stick to mixed effect model when random effect is insignficant?

Lao_feng — Tue, 13 Nov 2018 09:25:33 GMT

I am doing a logistic regression analysis with random intercept in the model to account for within cluster correlation.There are 3 levels in my data:

Level 1: individual subject

Level 2: Family (some subjects from the same family), the variable is fam_num

Level 3: Village, the variable clu_num

The model included country, age, gender, education and marrital status.

My codes are:

proc glimmix data=work0 NOCLPRINT;
class country (ref='Pakistan') clu_num fam_num age4g(ref='40~49') gender edu2g mar2g ;
model comb2g(ref='0')= country age4g gender edu2g mar2g /solution Link=logit dist=binary
random int /sub=clu_num;
random int /sub=fam_num (clu_num) ;
COVTEST GLM;
run;

I used 'COVTEST GLM' to check if the outcome is independent or not within clusters. The results I got are as follow:

The P value is 0.4115, suggesting the clustering effects are not statistically significant. In such case, can I remove the random effect from model, and use standard logistic regression? Thanks

Covariance Parameter Estimates
Cov Parm	Subject	Estimate	Standard Error
Intercept	clu_num	0.01724	0.02710
Intercept	fam_n(clu_nu)	0.01920	0.1512

Tests of Covariance Parameters Based on the Residual Pseudo-Likelihood
Label	DF	-2 Res Log P-Like	ChiSq	Pr > ChiSq	Note
Independence	2	10507	0.58	0.4115	MI

Re: Do I need to stick to mixed effect model when random effect is insignficant?

sld — Tue, 04 Dec 2018 04:47:06 GMT

Belatedly, my personal opinion is that the statistical model should mimic the experimental design. So if you have clusters in the experimental design, then you have variance components associated with those clusters, and so you should keep those variance components regardless of whether the estimates are statistically "significant".

Re: Do I need to stick to mixed effect model when random effect is insignficant?

BISTGP — Tue, 04 Dec 2018 22:24:15 GMT

I tend to agree with sld. Your model should reflect the data generating process as much as possible. That the random effects are not statistically significant at p < 0.05 is also a sample size issue. Also, and this is more tactics than science, but if you are submitting for publication it will be low-hanging fruit for a reviewer to ask about clustering, so now you have included it.

Re: Do I need to stick to mixed effect model when random effect is insignficant?

Lao_feng — Wed, 05 Dec 2018 00:44:09 GMT

Thank you all for your kind help !