Statistical programming, matrix languages, and more

Power Simulation for a Two Proportions Test

Accepted Solution Solved
Reply
Regular Contributor
Posts: 180
Accepted Solution

Power Simulation for a Two Proportions Test

Hello all,

 

I am trying to make a power analysis for a future clinical trial in which two proportions are to be compared. The analysis to be done is a non-inferiority analysis, i.e., it will be a one sided test for proportions difference, and the difference of proportions will be compared to a non-inferiority margin. The null hypothesis is P1-P2=-0.02, while the alternative hypothesis will be P1-P2>-0.02. The proportion in the first group (control) is assumed to be 8% while in the treatment group 6% (a smaller proportion is better).

 

I wrote this simulation, very much based on Rick Wicklin's example which I read in his book (that's why I publish this on this board, hoping that Rick will see this). The simulation ran and I got results, however, three problems occurred:

 

1. The results differ significantly from the results obtained from a sample size software, which used a formula rather than a simulation. The sample size in my simulation turned out to be much higher (about 300 subjects more overall).

 

2. I saved not only the P-Values of the non-inferiority test, but also the lower limit of the corresponding CI, which is a common practice which I saw in several articles. I expected the two methods to give the exact same results. But they don't. Not exactly.

 

3. I tried using PROC POWER to comapre the simulation results, but I keep getting an error saying: "NTOTAL is not available as a result option for TEST=." despite copying my code from the SAS documentation examples and changing the numbers only.

Can you kindly assist me in figuring out what is wrong with my simulation ?

 

Thank you in advance

 

%macro ODSOff(); 
	ods graphics off;
	ods exclude all;
	ods noresults;
%mend;

%macro ODSOn(); 
	ods graphics on;
	ods exclude none;
	ods results;
%mend;

/* Macro Variables */
%let NumSamples = 10000;  
%let nimargin = 0.02; /* Non-inferiority margin */
%let negnimargin = -0.02; /* Negative non-inferiority margin */
%let P_SOC = 0.08;    
%let P_Treatment = 0.06;  

/* 1. Simulating N samples of EACH group (Total = 2N) for each sample */
data PowerSizeSim(drop = i);
	call streaminit(321);
	Pc = &P_SOC;                                                        
	Pt = &P_Treatment;                                                      
	do N =  200 to 500 by 25 ;    /* N - sample size per group */
   		do SampleID = 1 to &NumSamples;
      		do i = 1 to N;
         		c = 1; x1 = rand("BERNOULLI", Pc); output;
         		c = 2; x1 = rand("BERNOULLI", Pt); output;
      		end;
   		end;
	end;
run;

data SimulatedData;
	set PowerSizeSim;
	if x1 = 0 then x2 = 1; else if x1=1 then x2 = 0;
	drop Pc Pt;
run;

/* 2. Compute Statistics */
%ODSOff 
proc freq data = SimulatedData;
	by N SampleID;
	table c*x2 / riskdiff(noninf margin=&nimargin method=fm) alpha=0.025;
	ods output PdiffNoninf = NITESTS;
run;
%ODSOn  


/* 3. Construct indicator var for obs that reject H0 */ 
data ResultsSize; 
	set NITESTS; 
	RejectH0 = (PValue <= 0.05);
	LowerCI = (LowerCL >= &negnimargin);
run; 

proc freq data=ResultsSize noprint; 
   by N; 
   tables RejectH0 / out = SimPower(where = (RejectH0 = 1));
   tables LowerCI /  out = SimPowerCI(where = (LowerCI = 1));
run;


/* 4. Output */
title2 'Simulated Power by Sample Size';
proc report data = SimPower;     
   column N PERCENT;
   define N / display 'Sample Size Per Group';            
   define PERCENT / display 'Simulated Power';
run;
title2;


/* PROC Power Validation */
proc power;
    twosamplefreq test = FM groupproportions = (0.08 0.06) nullproportiondiff = -0.02 alpha = 0.025
                  sides = U  power = 0.8 /*ntotal = .*/ NPERGROUP=.;
run;

 


Accepted Solutions
Solution
‎06-11-2017 03:11 AM
SAS Super FREQ
Posts: 3,752

Re: Power Simulation for a Two Proportions Test

The simulation part looks okay, so I would review the PROC FREQ / RISKDIFF(...) doc to make sure it is doing what you want. Also, why are you using alpha=0.025 in some places and PVALUE <= 0.05 in another?

 

 

View solution in original post


All Replies
Solution
‎06-11-2017 03:11 AM
SAS Super FREQ
Posts: 3,752

Re: Power Simulation for a Two Proportions Test

The simulation part looks okay, so I would review the PROC FREQ / RISKDIFF(...) doc to make sure it is doing what you want. Also, why are you using alpha=0.025 in some places and PVALUE <= 0.05 in another?

 

 

Regular Contributor
Posts: 180

Re: Power Simulation for a Two Proportions Test

Rick, thank you. I think you solved the mystery, well, one of them anyway. I specified alpha=0.025 in order to get a 95% CI, but I forgot to test the P-Value vs 0.025. I changed it now and the results of both the CI and the hypothesis testing are identical. The only thing that is still a mystery is why proc power doesn't want to work, but I guess that with a good simulation in hand, I don't really need it.

SAS Super FREQ
Posts: 3,752

Re: Power Simulation for a Two Proportions Test

Unfortunately, I do not know PROC POWER well enough to advise you. But you could try posting just that question to the SAS Statistical Procedures Community.

☑ This topic is solved.

Need further help from the community? Please ask a new question.

Discussion stats
  • 3 replies
  • 316 views
  • 2 likes
  • 2 in conversation