Non-overlapping CIs in probabilities after multinomial logit: statistically significant difference?

[mjkop56]

This link has directions for conducting a multinomial logit, including how to calculate predicted probabilities.

https://stats.idre.ucla.edu/sas/dae/multinomiallogistic-regression/

Question: If the 95% CIs of the predicted probabilities do NOT overlap, can we conclude there is a statistically significant difference (at 5% level) between the probabilities?

I understand if the CIs do overlap, they may or may not be statistically significantly different.

EXAMPLE BELOW

For the below screenshot for example, when type of program = 3 and SES = 1 the predicted probability is 0.2021 (95% CI: 0.08459, 0.3197) where as for type of program = 2 and SES = 3, the probability is 0.7009 (95% CI: 0.5709, 0.8309). Therefore they do not overlap, so can we conclude the probabilities are statistically significant at the 5% level?

[StatDave · Accepted solution]

No. Confidence intervals that overlap or not is not equivalent to a test of the difference. If you want to test pairwise differences (or ratios) among the levels of the variable specified in the LSMEANS statement on the mean (probability) scale rather than the log odds scale, then you can use the NLMeans macro as shown in this note. See the section titled "Multinomial Response — Using the NLMeans macro".