simulations of two clinics per district required such large (>>300) clinic sample sizes as to be impractical (implying 10,000 sample overall!) so these are not shown.
Both the Tukey-Kramer and the Bonferroni adjustments are appropriate given the extent of group size differences, but the Tukey-Kramer is more powerful (sensitive), requiring 360 fewer sample size to achieve the desired precision.
** example precision estimation for a random control trial after **;
** Stroup (2016) **;
** see http://support.sas.com/resources/papers/proceedings16/11663-2016.pdf **;
data rct;
** this is a SAS data step that creates a SAS data set that is called an 'exemplar' **;
** (or representative) data set in the above article to be used in conjunction with **;
** the glimmix procedure below to simulate the impacts of your assumptions **;
** on the precision (and yes, with more work, power) of your experimental design **;
infile cards;
input block @@;
** each block is an independent replication of the 3-treatment **;
** (control + 2 new treatments) group experimental design **;
** in your case if you make each block a randomly selected district **;
** then you get to estimate inter-district response variance for 'free' **;
** the double trailling @ in the input statement holds the observation from being output **;
do eu=1 to 3;
** iterates over each of the 3 experimental units in each (district?) block **;
** each eu (1-3) is a new experimental unit, which is permuted in this example **;
** the double trailing @ holds the observation from output until the end of dataline **;
input trt @@;
** current assumptions for success probabilities of the control (p1) and two **;
** treatments (p2,p3) are set here. These treatments are not as effective again as those assumed **;
** in previous simulations. By varying these and the eu size you can see what effect size difference **;
**(here I use 10% or 0.10 diff amongst treatments and the tukey-sidak **;
** adjustment for multiple comparisons) can be confidently detected at a given 95% level. **;
** I found this treatment effect size difference of 0.10 could just be detected at a 95 % C.L with a **;
** overall sample size of 2165 by changing these probabilites, setting eu size to 60 and re-running **;
** the exemplar analysis **;
p1=.30;p2=.40;p3=.50;
** p takes on the right value given the newly input treatment type. (trt1=1) =1 if trt =1, else = 0 **;
p=(trt=1)*p1+(trt=2)*p2+(trt=3)*p3;
** the ranuni(seed) function generates a uniform random number between 0 and 1 **;
** rounding 10 x this number to an integer and adding to 10 will uniformly randomly **;
** generate (and therefore facilitate simulation of the impact of) experimental unit **;
** sample sizes in the range 55 - 65, mean=60. Using the same seed reproduces the same **;
** psuedo-random sequence of sample sizes every time. This is my 'lucky' number! **;
** to change the cluster size assumption change the 55 to something else. To change the size variation**;
** change the 10 to something else. To use your own 'lucky' random seed change 09051958 to your **;
** own birthday or any other easy to remember number. There is no restriction on the number of digits.**;
** You can leave it blank for a new seed each time, but if you do, you will get a different, but equally **;
** varied set of experimental unit (clinic) samples sizes each time you run it, and sometimes that is a real **;
** nuisance **;
n=55+round(10*ranuni(09051958),1);
** mu is the expected number of positive outcomes from each experimental unit**;
mu=round(n*p,1);
** and the simulated experimental outcome is output to the exemplar data set **;
output;
** and this is done 3 times, one for each experimental unit **;
end;
** in the datalines below I have simulated 12 districts (blocks) chosen at random each with **;
** 3 clinics chosen at random for the trial. The treatments are allocated in all possible permutation **;
** orders, twice over the 12 blocks (districts). It is vitally important to vary treatments within block **;
** other designs that do not include this principle fail to have any useful precison **;
cards;
1 1 2 3
2 1 3 2
3 2 1 3
4 2 3 1
5 3 1 2
6 3 2 1
7 1 2 3
8 1 3 2
9 2 1 3
10 2 3 1
11 3 1 2
12 3 2 1
run;
proc glimmix data=rct;
class trt;
** this model statement form model y/n=x automatically invokes /dist=binomial link=logit **;
model mu/n=trt / ddfm=contain;
random intercept trt / subject=block;
** see the reference article by Stroup 2016 on how to make educated assumptions abut the **;
** group and treatment covariances. /hold tells glimmix not to estimate but hold covariance **;
** parameters to the values given below **;
parms (0.13)(0.10)/hold=1,2;
** this tests for strong evidence of a difference at the precision and sample size simulated **;
** Bonferroni is an appropriate multiple comparison adjustment when all differences are of interest. **;
** and group sizes are unequal by more than 20% of the geometric mean group size. Also, the **;
** tukey-cramer adjustment is also appropriate and more powerful in this situation. **;
lsmeans trt/diff cl adjust=tukey;
run;
| block |
0.1300 |
. |
| block |
0.1000 |
. |
| 1 |
2 |
-0.4292 |
0.1703 |
22 |
-2.52 |
0.0195 |
0.0492 |
0.05 |
-0.7824 |
-0.07592 |
-0.8571 |
-0.00128 |
| 1 |
3 |
-0.8520 |
0.1698 |
22 |
-5.02 |
<.0001 |
0.0001 |
0.05 |
-1.2040 |
-0.4999 |
-1.2784 |
-0.4255 |
| 2 |
3 |
-0.4228 |
0.1673 |
22 |
-2.53 |
0.0192 |
0.0485 |
0.05 |
-0.7698 |
-0.07577 |
-0.8431 |
-0.00245 |
This is 360 observations cheaper than the sample size implied by the bonferroni adjustment:
** example precision estimation for a random control trial after **;
** Stroup (2016) **;
** see http://support.sas.com/resources/papers/proceedings16/11663-2016.pdf **;
data rct;
** this is a SAS data step that creates a SAS data set that is called an 'exemplar' **;
** (or representative) data set in the above article to be used in conjunction with **;
** the glimmix procedure below to simulate the impacts of your assumptions **;
** on the precision (and yes, with more work, power) of your experimental design **;
infile cards;
input block @@;
** each block is an independent replication of the 3-treatment **;
** (control + 2 new treatments) group experimental design **;
** in your case if you make each block a randomly selected district **;
** then you get to estimate inter-district response variance for 'free' **;
** the double trailling @ in the input statement holds the observation from being output **;
do eu=1 to 3;
** iterates over each of the 3 experimental units in each (district?) block **;
** each eu (1-3) is a new experimental unit, which is permuted in this example **;
** the double trailing @ holds the observation from output until the end of dataline **;
input trt @@;
** current assumptions for success probabilities of the control (p1) and two **;
** treatments (p2,p3) are set here. These treatments are not as effective as those assumed **;
** in previous simulations. By varying these and the eu size you can see what effect size difference **;
**(here I use 10% or 0.10 diff between control and experimental treatments and the Bonferroni **;
** adjustment for multiple comparisons) can be confidently detected at a given 95% level. **;
** I found this treatment effect size difference of 0.10 could just be detected at a 95 % C.L with a **;
** overall sample size of 2525 by changing these probabilites and eu size and re-running the exemplar analysis **;
p1=.30;p2=.40;p3=.50;
** p takes on the right value given the newly input treatment type. (trt1=1) =1 if trt =1, else = 0 **;
p=(trt=1)*p1+(trt=2)*p2+(trt=3)*p3;
** the ranuni(seed) function generates a uniform random number between 0 and 1 **;
** rounding 10 x this number to an integer and adding to 10 will uniformly randomly **;
** generate (and therefore facilitate simulation of the impact of) experimental unit **;
** sample sizes in the range 65 - 75, mean=70. Using the same seed reproduces the same **;
** psuedo-random sequence of sample sizes every time. This is my 'lucky' number! **;
** to change the cluster size assumption change the 16 to something else. To change the size variation**;
** change the 10 to something else. To use your own 'lucky' random seed change 09051958 to your **;
** own birthday or any other easy to remember number. There is no restriction on the number of digits.**;
** You can leave it blank for a new seed each time, but if you do, you will get a different, but equally **;
** varied set of experimental unit (clinic) samples sizes each time you run it, and sometimes that is a real **;
** nuisance **;
n=65+round(10*ranuni(09051958),1);
** mu is the expected number of positive outcomes from each experimental unit**;
mu=round(n*p,1);
** and the simulated experimental outcome is output to the exemplar data set **;
output;
** and this is done 3 times, one for each experimental unit **;
end;
** in the datalines below I have simulated 12 districts (blocks) chosen at random each with **;
** 3 clinics chosen at random for the trial. The treatments are allocated in all possible permutation **;
** orders, twice over the 12 blocks (districts). It is vitally important to vary treatments within block **;
** other designs that do not include this principle fail to have any useful precison **;
cards;
1 1 2 3
2 1 3 2
3 2 1 3
4 2 3 1
5 3 1 2
6 3 2 1
7 1 2 3
8 1 3 2
9 2 1 3
10 2 3 1
11 3 1 2
12 3 2 1
run;
proc glimmix data=rct;
class trt;
** this model statement form model y/n=x automatically invokes /dist=binomial link=logit **;
model mu/n=trt / ddfm=contain;
random intercept trt / subject=block;
** see the reference article by Stroup 2016 on how to make educated assumptions abut the **;
** group and treatment covariances. /hold tells glimmix not to estimate but hold covariance **;
** parameters to the values given below **;
parms (0.13)(0.10)/hold=1,2;
** this tests for strong evidence of a difference at the precision and sample size simulated **;
** Bonferroni is an appropriate multiple comparison adjustment when all differences are of interest. **;
** and group sizes are unequal by more than 20% of the geometric mean group size. Also, the **;
** tukey-cramer adjustment is also appropriate and more powerful in this situation, but here the **;
** bonferroni adjustment is applied which results in an additional 360 sample size over that required **;
** for the justifiable tukey-kramer adjustment, all other assumption being equal.**;
lsmeans trt/diff cl adjust=bon;
run;
| block |
0.1300 |
. |
| block |
0.1000 |
. |
| 1 |
2 |
-0.4310 |
0.1651 |
22 |
-2.61 |
0.0160 |
0.0479 |
0.05 |
-0.7734 |
-0.08860 |
-0.8588 |
-0.00319 |
| 1 |
3 |
-0.8513 |
0.1646 |
22 |
-5.17 |
<.0001 |
0.0001 |
0.05 |
-1.1926 |
-0.5100 |
-1.2778 |
-0.4249 |
| 2 |
3 |
-0.4203 |
0.1624 |
22 |
-2.59 |
0.0168 |
0.0504 |
0.05 |
-0.7572 |
-0.08348 |
-0.8412 |
0.000548 |
