Power calculation using Proc Power

[AzizAA]

Hi,

I am doing a sample size calculation for Fisher's exact test. Here are the details:

5% probability of success in the control group.

85% probability of success in the intervention group.

For a sample size of 8 per group, proc power gives me a power of 0.851

Other software (R, nQuery and even simulation) gives 93%.

 Anyone knows why sas gives a different answer? My experimental units are very expensive and I would like to use no more than is absolutely necessary.

Thanks!

[PaigeMiller]

The TWOSAMPLEFREQ statement in PROC POWER allows you to determine sample sizes for Fisher's Exact Test. There is an example here, if you scroll down: https://documentation.sas.com/doc/en/pgmsascdc/9.4_3.4/statug/statug_power_syntax83.htm

[AzizAA]

Thanks. I used it but it gives (what I think) is an incorrect power.

Regards.

[PaigeMiller]

---

@AzizAA wrote:

Thanks. I used it but it gives (what I think) is an incorrect power.

---

I don't know what you mean. 

[AzizAA]

The power calculation that sas gives is incorrect. Has anyone come across this?

[PaigeMiller]

Just because you say it is incorrect doesn't mean I understand what has happened, what the results are, and why you think it is incorrect. Would you please explain what you did (SHOW US the code), what the results are (SHOW US a screen capture of the results) and in words state why you think it is incorrect.

For your future benefit, saying something is incorrect, and not providing additional details, never gets us closer to solving the problem. In the future, don't make us ask for additional details, just provide this information automatically.

[AzizAA]

All the information that is required is in the question after "Here are the details:".
Thanks!

[Ksharp]

Why not buy a ticket for sas support ?

https://support.sas.com/en/technical-support/submit-a-support-request.lang.html

[jiltao]

Can you share your PROC POWER code?

[AzizAA]

Here it is:

proc power;
   twosamplefreq  test=fisher
      groupproportions=(.05 .85)
      npergroup=8
      power=.
   ;
run;

[ballardw]

---

@AzizAA wrote:

Hi,

I am doing a sample size calculation for Fisher's exact test. Here are the details:

5% probability of success in the control group.

85% probability of success in the intervention group.

For a sample size of 8 per group, proc power gives me a power of 0.851

Other software (R, nQuery and even simulation) gives 93%.

 Anyone knows why sas gives a different answer? My experimental units are very expensive and I would like to use no more than is absolutely necessary.

Thanks!

---

Without out the code that you ran we can't tell if you provided the options correctly. If that is incorrect then the results are not reliable, much less which specific other software settings may have been used.

Is this the code that you ran:

proc power;
   twosamplefreq  test=fisher
      groupproportions=(.05 .85)
      npergroup=8
      power=.
   ;
run;

Which yields:

Fixed Scenario Elements
Distribution	Exact conditional
Method	Walters normal approximation
Group 1 Proportion	0.05
Group 2 Proportion	0.85
Sample Size per Group	8
Number of Sides	2
Alpha	0.05

Computed Power
Power
0.851

You say a Fishers Exact test, but one or two sided? upper or lower onesided? More than one test actually involved.

This gives a power of .912

proc power;
   twosamplefreq  test=fisher
      groupproportions=(.05 .85)
      npergroup=8
      sides= 1
      power=.
   ;
run;

So perhaps you forgot to mention the other requests were for one-sided tests???

Also if you want to worry about sample size, the provide power and request samples:

[AzizAA]

Thank you so much. You have used the code that I had (the two-sided test). I have tried several other power software and they all agree that the power is 93% for n=8 per group, so I am convinced that sas is alone in its calculation and incorrect. I'll buy a ticket to support at someone kindly suggested.

Best regards!

[FreelanceReinh · Accepted solution]

Hello @AzizAA,

While Fisher's exact test is exact, the power calculation for that test used by PROC POWER is just an approximation (see the formula in the documentation involving the normal distribution and the description referring to "a test with similar power properties," i.e., not even Fisher's exact test itself). It seems to me that in your particular case an exact computation would be preferable to that approximation. Given the small sample size this is fairly easy:

%let n=8;        /* sample size per group */
%let alpha=0.05; /* significance level */
%let p1=0.05;    /* success probability in group 1 */
%let p2=0.85;    /* success probability in group 2 */

/* Create a dataset with all (&n+1)**2-2 possible combinations of i=0, ..., &n successes (r=1) in group 1 (g=1)
   and j=0, ..., &n successes in group 2 (g=2), except the two extreme cases i=j=0 and i=j=&n */

data comb;
do i=0 to &n;
  do j=0 to &n;
    if i=j=0 | i=j=&n then continue;
    g=1;
    r=0; c=&n-i; output;
    r=1; c=i;    output;
    g=2;
    r=0; c=&n-j; output;
    r=1; c=j;    output;
  end;
end;
run;

/* Compute Fisher's exact test for each combination */

ods select none;
ods output FishersExact=fisher(where=(name1='XP2_FISH') keep=i j name1 nvalue1 rename=(nvalue1=p));
proc freq data=comb;
by i j;
tables g*r / chisq;
weight c;
run;
ods select all;

/* Compute power based on the joint distribution of two independent
   Bin(&n,&p1) and Bin(&n,&p2) distributed random variables */

data _null_;
set fisher end=last;
where .<p<=&alpha; /* The two extreme cases excluded above would not meet this condition anyway. */
power+pdf('binom',i,&p1,&n)*pdf('binom',j,&p2,&n);
if last then put power= 6.4;
run;

The result is indeed:

power=0.9345

[AzizAA]

Thanks for taking the time. That is great.

Have a good day!