topic Re: How to check model fit using PROC GENMOD in Statistical Procedures

How to check model fit using PROC GENMOD

Diels_O — Sat, 31 Dec 2022 08:27:10 GMT

Dear Professors, I met a problem when modeling modified Poisson Regression when using the PROC GENMOD and repeated statement, how should I assess the model fit? The response variable was a binary outcome.

proc genmod data = ITT1;
class enroll_order group(desc) D90_score_0to1(desc) IVT/PARAM = GLM;
model D90_score_0to1 = group age baseline_NIHSS IVT onset_to_randomization/ dist = poisson link = log covB corrb;
repeated subject = enroll_order/type = exch;
estimate "beta" group 1 -1/exp;
lsmeans group/diff exp cl;
run;

Re: How to check model fit using PROC GENMOD

Ksharp — Sat, 31 Dec 2022 09:28:16 GMT

You need to create a calibration plot .
@Rick_SAS wrote a blog about it.
It looks like a calibration plot can be used to check any model 's goodness of fit .

https://blogs.sas.com/content/iml/2019/02/20/easier-calibration-plot-sas.html
https://blogs.sas.com/content/iml/2018/05/14/calibration-plots-in-sas.html

Re: How to check model fit using PROC GENMOD

Ksharp — Sat, 31 Dec 2022 09:33:28 GMT

https://blogs.sas.com/content/iml/2018/05/16/decile-calibration-plots-sas.html
https://blogs.sas.com/content/iml/2020/11/23/decile-plots-in-sas.html

Re: How to check model fit using PROC GENMOD

Diels_O — Sat, 31 Dec 2022 12:35:24 GMT

Thank you for your generous help! However, it seems that the calibration plot works for continuous regressor, what if the regressors are binary?

Re: How to check model fit using PROC GENMOD

Diels_O — Sat, 31 Dec 2022 14:14:42 GMT

The instructions I'm following using the link above use the repeated statement with robust error variance (Zou G. A Modified Poisson Regression Approach to Prospective Studies with Binary Data. Am J Epidemiol 2004; 159(7):702-6.). However, according to The User's Guide for GENMOD which says that you do not get the Pearson chi-square and df ratio when you use a REPEATED statement. How can I check the goodness of fit in such case?

Re: How to check model fit using PROC GENMOD

Ksharp — Sun, 01 Jan 2023 08:22:49 GMT

Notice that @Rick_SAS using PROC LOGISTIC as an example ,the same as your binary Y variable.

Re: How to check model fit using PROC GENMOD

Ksharp — Sun, 01 Jan 2023 08:25:08 GMT

You can get the predicted Y value as Rick's blog showed .
For GEE or Mixed model you also could get these Y hat(a.k.a predicted Y) by OUTPUT statment,
since you have original Y and predicted Y ,you can plot a calibration plot to check model' s goodness of fit .
Maybe @Rick_SAS could give you a hand .

Re: How to check model fit using PROC GENMOD

Diels_O — Sun, 01 Jan 2023 14:36:14 GMT

Thank you very much and wish you a happy and prosperous 2023!

Re: How to check model fit using PROC GENMOD

StatDave — Sun, 01 Jan 2023 16:25:24 GMT

Just an added note that in addition to the plot, you can use the QIC statistic provided by GENMOD to compare competing models (models with differing predictors) and/or a generalized R-square statistic. Both are mentioned in the section of this note on GEE models, and the R-square statistic is further discussed and illustrated in this note. The first note also discusses how you can evaluate how well the model fits each observation and how to check the model form and link function.

Re: How to check model fit using PROC GENMOD

Diels_O — Wed, 04 Jan 2023 03:24:19 GMT

Thank you Dave for replying, it seems that the QIC provided a measurement that compared the model with different link/distrubution and working matrix. It seems that it cannot provide a test that provided a p value (e.g. Pearson Chi square) for the statistics of goodness of fit?