Quartz | Level 8

## Questions about testing significant differences in proportion.

Hello,

We tried to test a new question on our existing survey by asking same question in three different wordings(C group, T1 group and T 2 group) to see which one will yield the highest response rate.

Therefore, we sent total of 10,000 surveys out with evenly distributed among three groups(C=33.3333%, T1=33.3333% and T2=33.333%).

The table below shows that the # of responses and percentage for each group who answered the survey.

1)How to test significant differences in proportion among those three groups based on the observed results?

In other words, 34.9% for T2 is significantly higher than 33.4% for T1?  and 33.4% for T1 is significantly higher than 31.7% for C?

2) How the sample size(here we sent 10,000 surveys out, if we send out 20,000 or more) impact the test result?

3)How to interpret the results base on the SAS output?

Could you shed some lights?

Ethan

The FREQ Procedure

 TIMESTAMP Frequency Percent CumulativeFrequency CumulativePercent C 308 31.72 308 31.72 T1 324 33.37 632 65.09 T2 339 34.91 971 100.00
3 REPLIES 3
Barite | Level 11

## Re: Questions about testing significant differences in proportion.

1) Chi-square test of difference (chisq option in proc freq)
2) P value will decrease with greater sample size
3) P value < 0.05 means all groups are not equal in response rate. Right now the T groups stand out a little from the C group.

Fluorite | Level 6

## Re: Questions about testing significant differences in proportion.

Hello!

To test for significant differences in proportions among the C, T1, and T2 groups:

1. Use a chi-squared test for independence.
2. Calculate expected frequencies, chi-squared statistic, and degrees of freedom.
3. Compare chi-squared statistic with critical value at chosen significance level.
4. Larger sample sizes increase test power but don't guarantee meaningful differences.

Interpret SAS output by examining chi-squared statistic, degrees of freedom, and p-value. If p-value < chosen significance level, there's a significant difference between groups. Use PROC FREQ or PROC LOGISTIC in SAS for analysis.

Super User

## Re: Questions about testing significant differences in proportion.

Discussion stats
• 3 replies
• 354 views
• 5 likes
• 4 in conversation