topic Re: goodness of fit proc logistic in Statistical Procedures

goodness of fit proc logistic

mszommer — Thu, 11 Feb 2021 14:43:32 GMT

Hello,

I need help with choosing a goodness of fit test for binary logistic regression.

I have one independent variable, which is also categorical (binary) in nature. I got a not-so-fascinating c score of .54 and also a slightly moving-upward ROC curve.

I used the below code and obtained the below result:

proc logistic data=done.input_survey_v3 plots(maxpoints=none)=effect;
model Q6(event='1')=Q7 / aggregate scale=none;
run;

What does aggregate= option do? When must it be used?

Is Hosmer and Lemeshow GOF test meant for binary response data? I even tried that and got the same result - blanks.

I thank in advance for any help.

Regards,

Re: goodness of fit proc logistic

SteveDenham — Thu, 11 Feb 2021 14:10:32 GMT

A lot is going to depend on the data you are fitting. Can you share that dataset?

SteveDenham

Re: goodness of fit proc logistic

mszommer — Thu, 11 Feb 2021 14:50:22 GMT

Hello @SteveDenham,

Attached is a sample of my data. Q6 is my dependent var. and Q7 my independent.

Thank you for taking the time to look into my data.

Regards,

Re: goodness of fit proc logistic

StatDave — Thu, 11 Feb 2021 15:46:51 GMT

Since both variables are binary, you really have just a 2x2 table that defines two proportions. There is no need to fit a model for such data. You can simply compare those two proportions using the CHISQ option in PROC FREQ.

Concerning PROC LOGISTIC, the goodness of fit statistics have zero degrees of freedom because, with just a single binary predictor, the model is saturated and there are no degrees of freedom remaining. The AGGREGATE option is used in more complex models if the data is collected in subpopulations defined more precisely than by the covariates in the model as further described in this note.

Re: goodness of fit proc logistic

mszommer — Fri, 12 Feb 2021 07:13:22 GMT

Hello @StatDave,

Thank you for the explanation. It just slipped my mind that I could run a simple measures of association.

Having said that, I have seen several courses show fitting a model to such data (2 binary variables). Wouldn't the r2 always be too small in such cases? Obviously, the r2 value can't be improved as there are no independent variables to add.

I do look forward to your opinion.

Regards,

Mari

Re: goodness of fit proc logistic

StatDave — Fri, 12 Feb 2021 14:48:43 GMT

Unfortunately, R-square is not uniquely defined in generalized linear models like logistic models. There are many R-square statistics that have been devised but don't really have the clear "variance explained" interpretation of R-square in ordinary least-squares regression as from PROC REG and PROC GLM. Several R-square statistics are available with the GOF option in PROC LOGISTIC. Between this and since, with only one or two binary predictors there are only two or four populations, R-square is probably not a very useful statistic for assessing goodness of fit. If there is sufficient replication within the populations, then the Pearson and Deviance statistics are reasonable fit tests. In the case of two binary predictors, the model is not saturated if you don't include their interaction, so these tests and the additional tests provided by the GOF option can be useful.

Re: goodness of fit proc logistic

mszommer — Tue, 16 Feb 2021 08:39:11 GMT

Thank you @StatDave,. Really appreciate it.