Non-overlapping CIs in probabilities after multinomial logit: statistically significant difference?

mjkop56 — Tue, 28 Sep 2021 15:53:59 GMT

This link has directions for conducting a multinomial logit, including how to calculate predicted probabilities.

https://stats.idre.ucla.edu/sas/dae/multinomiallogistic-regression/

Question: If the 95% CIs of the predicted probabilities do NOT overlap, can we conclude there is a statistically significant difference (at 5% level) between the probabilities?

I understand if the CIs do overlap, they may or may not be statistically significantly different.

EXAMPLE BELOW

For the below screenshot for example, when type of program = 3 and SES = 1 the predicted probability is 0.2021 (95% CI: 0.08459, 0.3197) where as for type of program = 2 and SES = 3, the probability is 0.7009 (95% CI: 0.5709, 0.8309). Therefore they do not overlap, so can we conclude the probabilities are statistically significant at the 5% level?

Re: Non-overlapping CIs in probabilities after multinomial logit: statistically significant differen

StatDave — Tue, 28 Sep 2021 22:14:43 GMT

No. Confidence intervals that overlap or not is not equivalent to a test of the difference. If you want to test pairwise differences (or ratios) among the levels of the variable specified in the LSMEANS statement on the mean (probability) scale rather than the log odds scale, then you can use the NLMeans macro as shown in this note. See the section titled "Multinomial Response — Using the NLMeans macro".

topic Re: Non-overlapping CIs in probabilities after multinomial logit: statistically significant differen in Statistical Procedures

Non-overlapping CIs in probabilities after multinomial logit: statistically significant difference?

Re: Non-overlapping CIs in probabilities after multinomial logit: statistically significant differen