Goodness of Fit Multinomial SURVEYLOGISTIC

ray7 — Tue, 08 Apr 2025 17:03:31 GMT

Hi all, I’m currently working on a multinomial logistic regression using surveylogistic procedure. I’d like to assess the goodness-of-fit of the model. I noticed the output includes a likelihood ratio test, but I’m not sure whether that can be used as a measure of model fit in this context.

Any advice on how to assess model fit in this case?

Re: Goodness of Fit Multinomial SURVEYLOGISTIC

Season — Wed, 09 Apr 2025 07:48:10 GMT

As Heeringa et al. point out on page 269 of their book Amazon.com: Applied Survey Data Analysis (Chapman & Hall/CRC Statistics in the Social and Behavioral Sciences): 9780367736118: Heeringa, Steven G., West, Brady T., Berglund, Patricia A.: Books, complex survey invalidates the key assumptions under the ordinary likelihood ratio (LR) test, namely the kind of LR tests used in models built under simple random sampling.

However, Heeringa et al. also point out the presence of complex survey-adjusted versions of the LR test, including Tests for Regression Models Fitted to Survey Data - Lumley - 2014 - Australian & New Zealand Journal of Statistics - Wiley Online Library and AIC and BIC for modeling with complex survey data | Journal of Survey Statistics and Methodology | Oxford Academic. Moreover, as the name of the second paper cited in this paragraph indicates, complex survey-adjusted versions of Akaike information criterion (AIC) and Bayesian information criterion (BIC) have also been developed.

However, none of the aforementioned methods are available in any of SAS's SURVEY procedures like PROC SURVEYLOGISTIC. Instead, SAS uses a Rao-Scott correction for LR tests in complex surveys. Neither Heeringa et al. nor Taylor H. Lewis, author of Complex Survey Data Analysis with SAS | Taylor H. Lewis | Taylor & Fra, gave comments on whether the Rao-Scott correction is a valid method to assess the goodness-of-fit of logistic regression models built with survey data. Instead, both books suggested using the Wald's test as an alternative to the LR test when it comes to simulataneously testing whether a series of regression coefficients all equal to certain pre-specified values. But Wald's test is not a good choice of quantifying and comparing models' goodness-of-fit unless there is a model that fails it (i.e., the null assumption not rejected).

Summarizing my knowledge on this issue, I think a safer way to assess the goodness-of-fit of your model is to use complex survey-adjusted version of the AIC or BIC. Also, you can refer to the LR test proposed by Tests for Regression Models Fitted to Survey Data - Lumley - 2014 - Australian & New Zealand Journal of Statistics - Wiley Online Library. These methods, as Heeringa et al. point out, are available in the R package survey, but not in SAS.

As for the Rao-Scott adjusted version of LR tests provided by PROC SURVEYLOGISTIC, I think you can anyway retain its result as an alternative to the two methods I mentioned in the last paragraph, given that it is documented in SAS Help and is therefore endorsed by SAS's developers.

topic Goodness of Fit Multinomial SURVEYLOGISTIC in Statistical Procedures

Goodness of Fit Multinomial SURVEYLOGISTIC

Re: Goodness of Fit Multinomial SURVEYLOGISTIC