I am trying to make a power analysis for a future clinical trial in which two proportions are to be compared. The analysis to be done is a non-inferiority analysis, i.e., it will be a one sided test for proportions difference, and the difference of proportions will be compared to a non-inferiority margin. The null hypothesis is P1-P2=-0.02, while the alternative hypothesis will be P1-P2>-0.02. The proportion in the first group (control) is assumed to be 8% while in the treatment group 6% (a smaller proportion is better).
I wrote this simulation, very much based on Rick Wicklin's example which I read in his book (that's why I publish this on this board, hoping that Rick will see this). The simulation ran and I got results, however, three problems occurred:
1. The results differ significantly from the results obtained from a sample size software, which used a formula rather than a simulation. The sample size in my simulation turned out to be much higher (about 300 subjects more overall).
2. I saved not only the P-Values of the non-inferiority test, but also the lower limit of the corresponding CI, which is a common practice which I saw in several articles. I expected the two methods to give the exact same results. But they don't. Not exactly.
3. I tried using PROC POWER to comapre the simulation results, but I keep getting an error saying: "NTOTAL is not available as a result option for TEST=." despite copying my code from the SAS documentation examples and changing the numbers only.
Can you kindly assist me in figuring out what is wrong with my simulation ?
Thank you in advance
%macro ODSOff(); ods graphics off; ods exclude all; ods noresults; %mend; %macro ODSOn(); ods graphics on; ods exclude none; ods results; %mend; /* Macro Variables */ %let NumSamples = 10000; %let nimargin = 0.02; /* Non-inferiority margin */ %let negnimargin = -0.02; /* Negative non-inferiority margin */ %let P_SOC = 0.08; %let P_Treatment = 0.06; /* 1. Simulating N samples of EACH group (Total = 2N) for each sample */ data PowerSizeSim(drop = i); call streaminit(321); Pc = &P_SOC; Pt = &P_Treatment; do N = 200 to 500 by 25 ; /* N - sample size per group */ do SampleID = 1 to &NumSamples; do i = 1 to N; c = 1; x1 = rand("BERNOULLI", Pc); output; c = 2; x1 = rand("BERNOULLI", Pt); output; end; end; end; run; data SimulatedData; set PowerSizeSim; if x1 = 0 then x2 = 1; else if x1=1 then x2 = 0; drop Pc Pt; run; /* 2. Compute Statistics */ %ODSOff proc freq data = SimulatedData; by N SampleID; table c*x2 / riskdiff(noninf margin=&nimargin method=fm) alpha=0.025; ods output PdiffNoninf = NITESTS; run; %ODSOn /* 3. Construct indicator var for obs that reject H0 */ data ResultsSize; set NITESTS; RejectH0 = (PValue <= 0.05); LowerCI = (LowerCL >= &negnimargin); run; proc freq data=ResultsSize noprint; by N; tables RejectH0 / out = SimPower(where = (RejectH0 = 1)); tables LowerCI / out = SimPowerCI(where = (LowerCI = 1)); run; /* 4. Output */ title2 'Simulated Power by Sample Size'; proc report data = SimPower; column N PERCENT; define N / display 'Sample Size Per Group'; define PERCENT / display 'Simulated Power'; run; title2; /* PROC Power Validation */ proc power; twosamplefreq test = FM groupproportions = (0.08 0.06) nullproportiondiff = -0.02 alpha = 0.025 sides = U power = 0.8 /*ntotal = .*/ NPERGROUP=.; run;
Rick, thank you. I think you solved the mystery, well, one of them anyway. I specified alpha=0.025 in order to get a 95% CI, but I forgot to test the P-Value vs 0.025. I changed it now and the results of both the CI and the hypothesis testing are identical. The only thing that is still a mystery is why proc power doesn't want to work, but I guess that with a good simulation in hand, I don't really need it.
Registration is open! SAS is returning to Vegas for an AI and analytics experience like no other! Whether you're an executive, manager, end user or SAS partner, SAS Innovate is designed for everyone on your team. Register for just $495 by 12/31/2023.
If you are interested in speaking, there is still time to submit a session idea. More details are posted on the website.
Learn how to run multiple linear regression models with and without interactions, presented by SAS user Alex Chaplin.
Find more tutorials on the SAS Users YouTube channel.