BookmarkSubscribeRSS Feed
ray7
Calcite | Level 5

Hi all, I’m currently working on a multinomial logistic regression using surveylogistic procedure. I’d like to assess the goodness-of-fit of the model. I noticed the output includes a likelihood ratio test, but I’m not sure whether that can be used as a measure of model fit in this context.

Any advice on how to assess model fit in this case?

1 REPLY 1
Season
Barite | Level 11

As Heeringa et al. point out on page 269 of their book Amazon.com: Applied Survey Data Analysis (Chapman & Hall/CRC Statistics in the Social and Behavioral..., complex survey invalidates the key assumptions under the ordinary likelihood ratio (LR) test, namely the kind of LR tests used in models built under simple random sampling.

However, Heeringa et al. also point out the presence of complex survey-adjusted versions of the LR test, including Tests for Regression Models Fitted to Survey Data - Lumley - 2014 - Australian & New Zealand Journal... and AIC and BIC for modeling with complex survey data | Journal of Survey Statistics and Methodology | O.... Moreover, as the name of the second paper cited in this paragraph indicates, complex survey-adjusted versions of Akaike information criterion (AIC) and Bayesian information criterion (BIC) have also been developed.

However, none of the aforementioned methods are available in any of SAS's SURVEY procedures like PROC SURVEYLOGISTIC. Instead, SAS uses a Rao-Scott correction for LR tests in complex surveys. Neither Heeringa et al. nor Taylor H. Lewis, author of Complex Survey Data Analysis with SAS | Taylor H. Lewis | Taylor & Fra, gave comments on whether the Rao-Scott correction is a valid method to assess the goodness-of-fit of logistic regression models built with survey data. Instead, both books suggested using the Wald's test as an alternative to the LR test when it comes to simulataneously testing whether a series of regression coefficients all equal to certain pre-specified values. But Wald's test is not a good choice of quantifying and comparing models' goodness-of-fit unless there is a model that fails it (i.e., the null assumption not rejected).

Summarizing my knowledge on this issue, I think a safer way to assess the goodness-of-fit of your model is to use complex survey-adjusted version of the AIC or BIC. Also, you can refer to the LR test proposed by Tests for Regression Models Fitted to Survey Data - Lumley - 2014 - Australian & New Zealand Journal.... These methods, as Heeringa et al. point out, are available in the R package survey, but not in SAS.

As for the Rao-Scott adjusted version of LR tests provided by PROC SURVEYLOGISTIC, I think you can anyway retain its result as an alternative to the two methods I mentioned in the last paragraph, given that it is documented in SAS Help and is therefore endorsed by SAS's developers.

hackathon24-white-horiz.png

The 2025 SAS Hackathon has begun!

It's finally time to hack! Remember to visit the SAS Hacker's Hub regularly for news and updates.

Latest Updates

What is ANOVA?

ANOVA, or Analysis Of Variance, is used to compare the averages or means of two or more populations to better understand how they differ. Watch this tutorial for more.

Find more tutorials on the SAS Users YouTube channel.

Discussion stats
  • 1 reply
  • 577 views
  • 0 likes
  • 2 in conversation