A non-profit wants to know whether sending a communication improves constituent retention, and is it worth sending an expensive print piece over a free email?  The communication can be sent by email, snail mail, or not at all (i.e., three treatments).  The expected retention rate after a year in the untreated group is 93%.  Any measurable difference in retention  (0.5 percentage points or greater) would justify the costs of the snail mail. How would I calculate the sample size? If there were only two treatments (snail mail and no mail), I would do this: /* Is the improvement at least 1 percentage point? */ proc power;       twosamplefreq groupproportions=(0.93 0.940) test=pchi power=0.80 npergroup=. sides=1; /*  npergroup=7514 */ quit; Should I use LOGISTIC? In R it seems I would want pwr.chisq.test. ... View more

In SAS 9.1.3 I want to use PROC POWER for designing an A/B test for two treatments which produce proportions of success.  Based on my STAT 301 textbook, I was looking for two-sample t-test for proportions, but I didn't see it.  Should I be using "TwoSampleFreq test=pchi" instead? I expect the control group to have a 36.6% success rate and want the minimum number of samples needed to show the test group is not much worse, but I'm confused by the null and alternative hypothesis.  It is good if the two groups are the same because this means we can save money on a cheaper treatement, but normally the null hypothesis means no difference and it is "good" to reject the null hypothesis.  In other words, isn't it wrong to have "The null hypothesis is the two treatments have the same success rate" and try to not reject h0? ... View more