Hello all,
I am trying to make a power analysis for a future clinical trial in which two proportions are to be compared. The analysis to be done is a non-inferiority analysis, i.e., it will be a one sided test for proportions difference, and the difference of proportions will be compared to a non-inferiority margin. The null hypothesis is P1-P2=-0.02, while the alternative hypothesis will be P1-P2>-0.02. The proportion in the first group (control) is assumed to be 8% while in the treatment group 6% (a smaller proportion is better).
I wrote this simulation, very much based on Rick Wicklin's example which I read in his book (that's why I publish this on this board, hoping that Rick will see this). The simulation ran and I got results, however, three problems occurred:
1. The results differ significantly from the results obtained from a sample size software, which used a formula rather than a simulation. The sample size in my simulation turned out to be much higher (about 300 subjects more overall).
2. I saved not only the P-Values of the non-inferiority test, but also the lower limit of the corresponding CI, which is a common practice which I saw in several articles. I expected the two methods to give the exact same results. But they don't. Not exactly.
3. I tried using PROC POWER to comapre the simulation results, but I keep getting an error saying: "NTOTAL is not available as a result option for TEST=." despite copying my code from the SAS documentation examples and changing the numbers only.
Can you kindly assist me in figuring out what is wrong with my simulation ?
Thank you in advance
%macro ODSOff();
ods graphics off;
ods exclude all;
ods noresults;
%mend;
%macro ODSOn();
ods graphics on;
ods exclude none;
ods results;
%mend;
/* Macro Variables */
%let NumSamples = 10000;
%let nimargin = 0.02; /* Non-inferiority margin */
%let negnimargin = -0.02; /* Negative non-inferiority margin */
%let P_SOC = 0.08;
%let P_Treatment = 0.06;
/* 1. Simulating N samples of EACH group (Total = 2N) for each sample */
data PowerSizeSim(drop = i);
call streaminit(321);
Pc = &P_SOC;
Pt = &P_Treatment;
do N = 200 to 500 by 25 ; /* N - sample size per group */
do SampleID = 1 to &NumSamples;
do i = 1 to N;
c = 1; x1 = rand("BERNOULLI", Pc); output;
c = 2; x1 = rand("BERNOULLI", Pt); output;
end;
end;
end;
run;
data SimulatedData;
set PowerSizeSim;
if x1 = 0 then x2 = 1; else if x1=1 then x2 = 0;
drop Pc Pt;
run;
/* 2. Compute Statistics */
%ODSOff
proc freq data = SimulatedData;
by N SampleID;
table c*x2 / riskdiff(noninf margin=&nimargin method=fm) alpha=0.025;
ods output PdiffNoninf = NITESTS;
run;
%ODSOn
/* 3. Construct indicator var for obs that reject H0 */
data ResultsSize;
set NITESTS;
RejectH0 = (PValue <= 0.05);
LowerCI = (LowerCL >= &negnimargin);
run;
proc freq data=ResultsSize noprint;
by N;
tables RejectH0 / out = SimPower(where = (RejectH0 = 1));
tables LowerCI / out = SimPowerCI(where = (LowerCI = 1));
run;
/* 4. Output */
title2 'Simulated Power by Sample Size';
proc report data = SimPower;
column N PERCENT;
define N / display 'Sample Size Per Group';
define PERCENT / display 'Simulated Power';
run;
title2;
/* PROC Power Validation */
proc power;
twosamplefreq test = FM groupproportions = (0.08 0.06) nullproportiondiff = -0.02 alpha = 0.025
sides = U power = 0.8 /*ntotal = .*/ NPERGROUP=.;
run;
