topic Re: Osius-Rojek test for goodness of fit for PROC LOGISTIC in Statistical Procedures

Osius-Rojek test for goodness of fit for PROC LOGISTIC

blund — Thu, 31 Dec 2020 03:19:43 GMT

In SAS 9.4 / Viya 3.4 documentation for PROC LOGISTIC the formula for the variance of the Pearson chi-square statistic is discussed in the section entitled “Osius-Rojek Test”. See page 5858 of SAS/STAT® 15.1 User’s Guide The LOGISTIC Procedure (2018). PROC LOGISTIC computes Osius-Rojek for the training data set when "/ GOF" is added to the MODEL statement. Is this same O-R formula applicable to a validation data set?

NOTE: Osius-Rojek is not computed as part of the FITSTAT for a data set that is scored via SCORE DATA = VALIDATION OUT = SCORED FITSTAT;

Added Note:

The O-R formula for variance on page 5858 is not the same as the formula in Hosmer, Lemeshow, Sturdivant Applied Logistic Regression, Third Edition on page 203. In the SAS document on page 5858 there is a term c'Vc which is subtracted.

Re: Osius-Rojek test for goodness of fit for PROC LOGISTIC

blund — Wed, 14 Apr 2021 18:07:22 GMT

HLS (Hosmer, Lemeshow, Sturdivant Applied Logistic Regression) on pages 164-165 present an alternative z-statistic to the O-R z-statistic (Osius-Rojek) for goodness of fit measurement. Both z‑statistics are intended to be computed on the Training Dataset.

Let J be the number of profiles (distinct combinations of values of the x’s). Let XP2 be the Pearson chi‑square statistic.

The HLS z-statistic only differs in the numerator from the O-R z-statistic. Instead of the O-R numerator of XP2 - J, the HLS version subtracts the degrees of freedom (parameters K plus 1) from J. So, the HLS numerator becomes XP2 - (J - K - 1).

The formula for the denominator, as given in HLS pages 164-165, looks very different than the formula for O-R as given in SAS PROC LOGISTIC documentation given in SAS/STAT 15.1 documentation page 5858. However, this two formulas produce the same answer. (https://documentation.sas.com/api/collections/pgmsascdc/9.4_3.4/docsets/statug/content/statug.pdf?locale=en#nameddest=titlepage)

If a user wants to utilize the (HLS z-statistic)2 for measuring goodness of fit on the training group, it is easy to obtain this value from the (O-R z-statistic)2 as reported by PROC LOGISTIC. Here is the process:

Record the Pearson chi-square XP2 and (O-R z-statistic)2

Compute two numbers:

Num1 = (XP2 - J + K + 1) and Num2 = (XP2 - J)

Then (HLS z-statistic)2 is given by

(HLS z-statistic)2 = (O-R z-statistic)2 * (Num1/Num2)2

The (HLS z-statistic)2 may be less than or greater than (O-R z-statistic)2. Examples can be constructed for (XP2- J + K + 1)2 > (XP2 - J)2 and for (XP2 - J + K + 1)2 < (XP2 - J)2

Re: Osius-Rojek test for goodness of fit for PROC LOGISTIC

blund — Wed, 14 Apr 2021 18:12:30 GMT

In my REPLY, please read XP2 as XP**2. Read (O-R z-statistic)2 as (O-R z-statistic)**2 and (HLS z-statistic)2 as (HLS z-statistic)**2, etc.