topic Re: Questions about testing significant differences in proportion. in Statistical Procedures

Questions about testing significant differences in proportion.

t75wez1 — Mon, 12 Feb 2024 18:48:35 GMT

Hello,

We tried to test a new question on our existing survey by asking same question in three different wordings(C group, T1 group and T 2 group) to see which one will yield the highest response rate.

Therefore, we sent total of 10,000 surveys out with evenly distributed among three groups(C=33.3333%, T1=33.3333% and T2=33.333%).

The table below shows that the # of responses and percentage for each group who answered the survey.

1)How to test significant differences in proportion among those three groups based on the observed results?

In other words, 34.9% for T2 is significantly higher than 33.4% for T1? and 33.4% for T1 is significantly higher than 31.7% for C?

2) How the sample size(here we sent 10,000 surveys out, if we send out 20,000 or more) impact the test result?

3)How to interpret the results base on the SAS output?

Could you shed some lights?

Thanks for your help in advance.

Ethan

The FREQ Procedure

TIMESTAMP	Frequency	Percent	Cumulative Frequency	Cumulative Percent
C	308	31.72	308	31.72
T1	324	33.37	632	65.09
T2	339	34.91	971	100.00

Re: Questions about testing significant differences in proportion.

pink_poodle — Wed, 14 Feb 2024 04:32:45 GMT

1) Chi-square test of difference (chisq option in proc freq)
2) P value will decrease with greater sample size
3) P value < 0.05 means all groups are not equal in response rate. Right now the T groups stand out a little from the C group.

Re: Questions about testing significant differences in proportion.

jacksmith1 — Wed, 14 Feb 2024 05:41:35 GMT

Hello!

To test for significant differences in proportions among the C, T1, and T2 groups:

Use a chi-squared test for independence.
Calculate expected frequencies, chi-squared statistic, and degrees of freedom.
Compare chi-squared statistic with critical value at chosen significance level.
Larger sample sizes increase test power but don't guarantee meaningful differences.

Interpret SAS output by examining chi-squared statistic, degrees of freedom, and p-value. If p-value < chosen significance level, there's a significant difference between groups. Use PROC FREQ or PROC LOGISTIC in SAS for analysis.

Re: Questions about testing significant differences in proportion.

Ksharp — Wed, 14 Feb 2024 07:06:35 GMT

https://support.sas.com/kb/62/362.html
https://support.sas.com/kb/37/344.html
https://support.sas.com/kb/37/228.html