How to check model fit using PROC GENMOD

[Diels_O]

Dear Professors, I met a problem when modeling modified Poisson Regression when using the PROC GENMOD and repeated statement, how should I assess the model fit? The response variable was a binary outcome.

proc genmod data = ITT1;
class enroll_order group(desc) D90_score_0to1(desc) IVT/PARAM = GLM;
model D90_score_0to1 = group age baseline_NIHSS IVT onset_to_randomization/ dist = poisson link = log covB corrb;
repeated subject = enroll_order/type = exch;
estimate "beta" group 1 -1/exp;
lsmeans group/diff exp cl;
run;

[Ksharp]

You need to create a calibration plot .
@Rick_SAS wrote a blog about it.
It looks like a calibration plot can be used to check any model 's goodness of fit .

https://blogs.sas.com/content/iml/2019/02/20/easier-calibration-plot-sas.html
https://blogs.sas.com/content/iml/2018/05/14/calibration-plots-in-sas.html

[Ksharp]

https://blogs.sas.com/content/iml/2018/05/16/decile-calibration-plots-sas.html
https://blogs.sas.com/content/iml/2020/11/23/decile-plots-in-sas.html

[Diels_O]

Thank you for your generous help! However, it seems that the calibration plot works for continuous regressor, what if the regressors are binary?

[Ksharp]

Notice that @Rick_SAS using PROC LOGISTIC as an example ,the same as your binary Y variable.

[Diels_O]

The instructions I'm following using the link above use the repeated statement with robust error variance (Zou G. A Modified Poisson Regression Approach to Prospective Studies with Binary Data. Am J Epidemiol 2004; 159(7):702-6.). However, according to The User's Guide for GENMOD which says that you do not get the Pearson chi-square and df ratio when you use a REPEATED statement. How can I check the goodness of fit in such case?

[Ksharp · Accepted solution]

You can get the predicted Y value as Rick's blog showed .
For GEE or Mixed model you also could get these Y hat(a.k.a predicted Y) by OUTPUT statment,
since you have original Y and predicted Y ,you can plot a calibration plot to check model' s goodness of fit .
Maybe @Rick_SAS could give you a hand .

[Diels_O]

Thank you very much and wish you a happy and prosperous 2023!

[StatDave]

Just an added note that in addition to the plot, you can use the QIC statistic provided by GENMOD to compare competing models (models with differing predictors) and/or a generalized R-square statistic. Both are mentioned in the section of this note on GEE models, and the R-square statistic is further discussed and illustrated in this note. The first note also discusses how you can evaluate how well the model fits each observation and how to check the model form and link function. 

[Diels_O]

Thank you Dave for replying, it seems that the QIC provided a measurement that compared the model with different link/distrubution and working matrix. It seems that it cannot provide a test that provided a p value (e.g. Pearson Chi square) for the statistics of goodness of fit?