2 weeks ago
I am trying to plan a study, in which I wish to compare two "rare proportions".
It is assumed that with the current treatment (the standard of care) , the proportion of events (bad events) is 7%. It is also assumed that with the new treatment this proportion goes down to 4%. The expected difference is 3%. An effect of 3% might not sound like much statistically, but clinicians claim it is very significant clinically.
I've tried calculating the sample size for the two proportion test (Fisher's exact test) and got that for power of 80% and significance level of 5%, the sample size is 971 per group, which is quite a lot. This sample size will give me the desired power to test the hypothesis that P1-P2=0.
Alternatively, I've calculated and found that under these assumptions, the OR is 1.8. The sample size to test the hypothesis that OR=1, under the same power and significance level, is 545 per group - far less !
Now I am confused and looking for the "downfall". My questions to you are:
1. Using SAS - how do I run an hypothesis testing for the OR = 1 vs OR >1 or OR NE 1 ? I would like to write a simulation so I need to know how to do it. Can I simply use Logistic Regression ?
2. What is the disadvantage of using the OR instead of the proportion difference ? There has to be a catch, I mean, why would anyone test for proportion difference if you can bypass it and test for the OR with a much smaller sample size ? There has to be a disadvantage I currently can't think about. Do you think that the FDA will accept using the OR instead of the difference or the common Fisher's exact test ?
3. Is there another approach to handle this situation of rare events, in which clinicians say 3% is amazingly important (each event is hard medically), statistically it's a small effect, and samples of ~1000 per group are way to expensive for a company to pay ? What would you recommend ? Maybe Bayesian Statistics ? Any advice will be mostly appreciated !
Thank you !