Diagnostics for mixed effects logistic regression

[jgreenberg321]

Hello All,

I am fitting a mixed effects logistic regression model using proc glimmix to model post-operative surgical complications. I am interested in running model diagnostics, particularly to evaluate for any significant non-linearity in continuous predictors (e.g. relationship between age and log-odds of the outcome). I have had difficulty generating relevant plots using the glimmix procedure. Does anyone have any experience with this, or know of any good reference articles that might be helpful? All of the usefulf articles I've found on mixed effect model diagnostics in SAS are for linear models. For reference, my code setup is below. Thank you!

proc glimmix data=test method=laplace;

class hospitalid Race_Group insurance female proc_type;

model complication (event='1') = age comorbidity_index Race_Group insurance female proc_type/S

CL DIST=binary link=logit ;

output out=out pred(noblup ilink)=expected_prob pred(blup ilink)=predicted_prob;

random intercept/subject=hospitalid_index ;

covtest/wald;

run;

[PaigeMiller]

If you are worried about polynomial effects, for example is Age**2 a good predictor, you can simply put it into the model and see if it produces a significant effect. For more general non-linearity, you might have to look at the Pearson residuals plotted against Age.

[jgreenberg321]

Thanks. I'll try that, along with other transformations (e.g. square root). I suppose we could use something like an AIC value to judge if the model with the transformed variable is better than the untransformed? 

I imagine we could also just use sgplot to plot a loess cuve in the residual plot? Thanks again. 

[jgreenberg321]

Just to follow-up on this: do you happen to know how to conduct a likelihood ratio test using proc Glimmix? I see from the Glimmix documentation that is possible but I can't find any examples of exactly how to do that. That would be helpful for testing the need for a polynomial term.  Is that something that would be done through the "General" term? Example code is below.  Thank you!

proc glimmix data=test method=laplace ;

class hospitalid Race_Group insurance female proc_group (ref='1');

model major_comp (event='1') =age age_square confounder  Race_Group insurance female proc_group/solution

CL DIST=binary link=logit;

random intercept/subject=hospitalid;

run;

[SteveDenham · Accepted solution]

The type III tests for fixed effects are all likelihood ratio based/equivalent.  GLIMMIX has a COVTEST option that enables likelihood ratio tests for random effects.

I would tend to look at AICC values, rather than p values from the type III tests, to see if the enhanced model provided an improvement in the relative likelihood of retaining the information in the model with the smallest AICC.

This is behind a paywall, but is easily accessible for ASA members:

Stroup, W., Claassen, E. Pseudo-Likelihood or Quadrature? What We Thought We Knew, What We Think We Know, and What We Are Still Trying to Figure Out. JABES (2020). https://doi.org/10.1007/s13253-020-00402-6

In this paper, the authors point out some advantages to using the default RSPL method as opposed to quadrature methods and talk a bit about graphical methods for checking assumptions (nothing really good).  The point here is that if you do switch to the default RSPL method, you will no longer get information criteria in the output.  You will need to run the model twice, saving the final -2 log likelihood from the full model and the nested model, and then calculate the log likelihood change (chi squared) and associated probability.

SteveDenham

[jgreenberg321]

Thank you for explaining. Just to make sure I understand, the type 3 fixed effects values are equivalent to the LR test (i.e. testing the model with vs. without the added variable)? 

Fortunately in my example, the LR difference between models test aligns with the AIC change between the more complex and simpler models. Unfortunately, I was not able to access that paper, though can relate to the sentiment of lacking good measures for assumption checking.