BookmarkSubscribeRSS Feed
t75wez1
Quartz | Level 8

Hello,

We tried to test a new question on our existing survey by asking same question in three different wordings(C group, T1 group and T 2 group) to see which one will yield the highest response rate.

 

Therefore, we sent total of 10,000 surveys out with evenly distributed among three groups(C=33.3333%, T1=33.3333% and T2=33.333%).

The table below shows that the # of responses and percentage for each group who answered the survey.

 

1)How to test significant differences in proportion among those three groups based on the observed results?

 

In other words, 34.9% for T2 is significantly higher than 33.4% for T1?  and 33.4% for T1 is significantly higher than 31.7% for C?

 

2) How the sample size(here we sent 10,000 surveys out, if we send out 20,000 or more) impact the test result?

 

3)How to interpret the results base on the SAS output?

 

Could you shed some lights?

 

Thanks for your help in advance.

Ethan

 

The FREQ Procedure

TIMESTAMP

Frequency

Percent

Cumulative
Frequency

Cumulative
Percent

C

308

31.72

308

31.72

T1

324

33.37

632

65.09

T2

339

34.91

971

100.00

3 REPLIES 3
pink_poodle
Barite | Level 11

1) Chi-square test of difference (chisq option in proc freq)
2) P value will decrease with greater sample size
3) P value < 0.05 means all groups are not equal in response rate. Right now the T groups stand out a little from the C group.

jacksmith1
Fluorite | Level 6

Hello!

To test for significant differences in proportions among the C, T1, and T2 groups:

  1. Use a chi-squared test for independence.
  2. Calculate expected frequencies, chi-squared statistic, and degrees of freedom.
  3. Compare chi-squared statistic with critical value at chosen significance level.
  4. Larger sample sizes increase test power but don't guarantee meaningful differences.

Interpret SAS output by examining chi-squared statistic, degrees of freedom, and p-value. If p-value < chosen significance level, there's a significant difference between groups. Use PROC FREQ or PROC LOGISTIC in SAS for analysis.

 
 

sas-innovate-2024.png

Don't miss out on SAS Innovate - Register now for the FREE Livestream!

Can't make it to Vegas? No problem! Watch our general sessions LIVE or on-demand starting April 17th. Hear from SAS execs, best-selling author Adam Grant, Hot Ones host Sean Evans, top tech journalist Kara Swisher, AI expert Cassie Kozyrkov, and the mind-blowing dance crew iLuminate! Plus, get access to over 20 breakout sessions.

 

Register now!

What is ANOVA?

ANOVA, or Analysis Of Variance, is used to compare the averages or means of two or more populations to better understand how they differ. Watch this tutorial for more.

Find more tutorials on the SAS Users YouTube channel.

Discussion stats
  • 3 replies
  • 346 views
  • 5 likes
  • 4 in conversation