simulations of two clinics per district required such large (>>300) clinic sample sizes as to be impractical (implying 10,000 sample overall!) so these are not shown.
Both the Tukey-Kramer and the Bonferroni adjustments are appropriate given the extent of group size differences, but the Tukey-Kramer is more powerful (sensitive), requiring 360 fewer sample size to achieve the desired precision.
** example precision estimation for a random control trial after **;
** Stroup (2016) **;
** see http://support.sas.com/resources/papers/proceedings16/11663-2016.pdf **;
data rct;
** this is a SAS data step that creates a SAS data set that is called an 'exemplar' **;
** (or representative) data set in the above article to be used in conjunction with **;
** the glimmix procedure below to simulate the impacts of your assumptions **;
** on the precision (and yes, with more work, power) of your experimental design **;
infile cards;
input block @@;
** each block is an independent replication of the 3-treatment **;
** (control + 2 new treatments) group experimental design **;
** in your case if you make each block a randomly selected district **;
** then you get to estimate inter-district response variance for 'free' **;
** the double trailling @ in the input statement holds the observation from being output **;
do eu=1 to 3;
** iterates over each of the 3 experimental units in each (district?) block **;
** each eu (1-3) is a new experimental unit, which is permuted in this example **;
** the double trailing @ holds the observation from output until the end of dataline **;
input trt @@;
** current assumptions for success probabilities of the control (p1) and two **;
** treatments (p2,p3) are set here. These treatments are not as effective again as those assumed **;
** in previous simulations. By varying these and the eu size you can see what effect size difference **;
**(here I use 10% or 0.10 diff amongst treatments and the tukey-sidak **;
** adjustment for multiple comparisons) can be confidently detected at a given 95% level. **;
** I found this treatment effect size difference of 0.10 could just be detected at a 95 % C.L with a **;
** overall sample size of 2165 by changing these probabilites, setting eu size to 60 and re-running **;
** the exemplar analysis **;
p1=.30;p2=.40;p3=.50;
** p takes on the right value given the newly input treatment type. (trt1=1) =1 if trt =1, else = 0 **;
p=(trt=1)*p1+(trt=2)*p2+(trt=3)*p3;
** the ranuni(seed) function generates a uniform random number between 0 and 1 **;
** rounding 10 x this number to an integer and adding to 10 will uniformly randomly **;
** generate (and therefore facilitate simulation of the impact of) experimental unit **;
** sample sizes in the range 55 - 65, mean=60. Using the same seed reproduces the same **;
** psuedo-random sequence of sample sizes every time. This is my 'lucky' number! **;
** to change the cluster size assumption change the 55 to something else. To change the size variation**;
** change the 10 to something else. To use your own 'lucky' random seed change 09051958 to your **;
** own birthday or any other easy to remember number. There is no restriction on the number of digits.**;
** You can leave it blank for a new seed each time, but if you do, you will get a different, but equally **;
** varied set of experimental unit (clinic) samples sizes each time you run it, and sometimes that is a real **;
** nuisance **;
n=55+round(10*ranuni(09051958),1);
** mu is the expected number of positive outcomes from each experimental unit**;
mu=round(n*p,1);
** and the simulated experimental outcome is output to the exemplar data set **;
output;
** and this is done 3 times, one for each experimental unit **;
end;
** in the datalines below I have simulated 12 districts (blocks) chosen at random each with **;
** 3 clinics chosen at random for the trial. The treatments are allocated in all possible permutation **;
** orders, twice over the 12 blocks (districts). It is vitally important to vary treatments within block **;
** other designs that do not include this principle fail to have any useful precison **;
cards;
1 1 2 3
2 1 3 2
3 2 1 3
4 2 3 1
5 3 1 2
6 3 2 1
7 1 2 3
8 1 3 2
9 2 1 3
10 2 3 1
11 3 1 2
12 3 2 1
run;
proc glimmix data=rct;
class trt;
** this model statement form model y/n=x automatically invokes /dist=binomial link=logit **;
model mu/n=trt / ddfm=contain;
random intercept trt / subject=block;
** see the reference article by Stroup 2016 on how to make educated assumptions abut the **;
** group and treatment covariances. /hold tells glimmix not to estimate but hold covariance **;
** parameters to the values given below **;
parms (0.13)(0.10)/hold=1,2;
** this tests for strong evidence of a difference at the precision and sample size simulated **;
** Bonferroni is an appropriate multiple comparison adjustment when all differences are of interest. **;
** and group sizes are unequal by more than 20% of the geometric mean group size. Also, the **;
** tukey-cramer adjustment is also appropriate and more powerful in this situation. **;
lsmeans trt/diff cl adjust=tukey;
run;
Number of Observations Read
36
Number of Observations Used
36
Number of Events
869
Number of Trials
2165
Covariance Parameter Estimates
Cov Parm
Subject
Estimate
Standard Error
Intercept
block
0.1300
.
trt
block
0.1000
.
Differences of trt Least Squares Means Adjustment for Multiple Comparisons: Tukey-Kramer
trt
_trt
Estimate
Standard Error
DF
t Value
Pr > |t|
Adj P
Alpha
Lower
Upper
Adj Lower
Adj Upper
1
2
-0.4292
0.1703
22
-2.52
0.0195
0.0492
0.05
-0.7824
-0.07592
-0.8571
-0.00128
1
3
-0.8520
0.1698
22
-5.02
<.0001
0.0001
0.05
-1.2040
-0.4999
-1.2784
-0.4255
2
3
-0.4228
0.1673
22
-2.53
0.0192
0.0485
0.05
-0.7698
-0.07577
-0.8431
-0.00245
This is 360 observations cheaper than the sample size implied by the bonferroni adjustment:
** example precision estimation for a random control trial after **;
** Stroup (2016) **;
** see http://support.sas.com/resources/papers/proceedings16/11663-2016.pdf **;
data rct;
** this is a SAS data step that creates a SAS data set that is called an 'exemplar' **;
** (or representative) data set in the above article to be used in conjunction with **;
** the glimmix procedure below to simulate the impacts of your assumptions **;
** on the precision (and yes, with more work, power) of your experimental design **;
infile cards;
input block @@;
** each block is an independent replication of the 3-treatment **;
** (control + 2 new treatments) group experimental design **;
** in your case if you make each block a randomly selected district **;
** then you get to estimate inter-district response variance for 'free' **;
** the double trailling @ in the input statement holds the observation from being output **;
do eu=1 to 3;
** iterates over each of the 3 experimental units in each (district?) block **;
** each eu (1-3) is a new experimental unit, which is permuted in this example **;
** the double trailing @ holds the observation from output until the end of dataline **;
input trt @@;
** current assumptions for success probabilities of the control (p1) and two **;
** treatments (p2,p3) are set here. These treatments are not as effective as those assumed **;
** in previous simulations. By varying these and the eu size you can see what effect size difference **;
**(here I use 10% or 0.10 diff between control and experimental treatments and the Bonferroni **;
** adjustment for multiple comparisons) can be confidently detected at a given 95% level. **;
** I found this treatment effect size difference of 0.10 could just be detected at a 95 % C.L with a **;
** overall sample size of 2525 by changing these probabilites and eu size and re-running the exemplar analysis **;
p1=.30;p2=.40;p3=.50;
** p takes on the right value given the newly input treatment type. (trt1=1) =1 if trt =1, else = 0 **;
p=(trt=1)*p1+(trt=2)*p2+(trt=3)*p3;
** the ranuni(seed) function generates a uniform random number between 0 and 1 **;
** rounding 10 x this number to an integer and adding to 10 will uniformly randomly **;
** generate (and therefore facilitate simulation of the impact of) experimental unit **;
** sample sizes in the range 65 - 75, mean=70. Using the same seed reproduces the same **;
** psuedo-random sequence of sample sizes every time. This is my 'lucky' number! **;
** to change the cluster size assumption change the 16 to something else. To change the size variation**;
** change the 10 to something else. To use your own 'lucky' random seed change 09051958 to your **;
** own birthday or any other easy to remember number. There is no restriction on the number of digits.**;
** You can leave it blank for a new seed each time, but if you do, you will get a different, but equally **;
** varied set of experimental unit (clinic) samples sizes each time you run it, and sometimes that is a real **;
** nuisance **;
n=65+round(10*ranuni(09051958),1);
** mu is the expected number of positive outcomes from each experimental unit**;
mu=round(n*p,1);
** and the simulated experimental outcome is output to the exemplar data set **;
output;
** and this is done 3 times, one for each experimental unit **;
end;
** in the datalines below I have simulated 12 districts (blocks) chosen at random each with **;
** 3 clinics chosen at random for the trial. The treatments are allocated in all possible permutation **;
** orders, twice over the 12 blocks (districts). It is vitally important to vary treatments within block **;
** other designs that do not include this principle fail to have any useful precison **;
cards;
1 1 2 3
2 1 3 2
3 2 1 3
4 2 3 1
5 3 1 2
6 3 2 1
7 1 2 3
8 1 3 2
9 2 1 3
10 2 3 1
11 3 1 2
12 3 2 1
run;
proc glimmix data=rct;
class trt;
** this model statement form model y/n=x automatically invokes /dist=binomial link=logit **;
model mu/n=trt / ddfm=contain;
random intercept trt / subject=block;
** see the reference article by Stroup 2016 on how to make educated assumptions abut the **;
** group and treatment covariances. /hold tells glimmix not to estimate but hold covariance **;
** parameters to the values given below **;
parms (0.13)(0.10)/hold=1,2;
** this tests for strong evidence of a difference at the precision and sample size simulated **;
** Bonferroni is an appropriate multiple comparison adjustment when all differences are of interest. **;
** and group sizes are unequal by more than 20% of the geometric mean group size. Also, the **;
** tukey-cramer adjustment is also appropriate and more powerful in this situation, but here the **;
** bonferroni adjustment is applied which results in an additional 360 sample size over that required **;
** for the justifiable tukey-kramer adjustment, all other assumption being equal.**;
lsmeans trt/diff cl adjust=bon;
run;
Number of Observations Read
36
Number of Observations Used
36
Number of Events
1013
Number of Trials
2525
Covariance Parameter Estimates
Cov Parm
Subject
Estimate
Standard Error
Intercept
block
0.1300
.
trt
block
0.1000
.
Differences of trt Least Squares Means Adjustment for Multiple Comparisons: Bonferroni
trt
_trt
Estimate
Standard Error
DF
t Value
Pr > |t|
Adj P
Alpha
Lower
Upper
Adj Lower
Adj Upper
1
2
-0.4310
0.1651
22
-2.61
0.0160
0.0479
0.05
-0.7734
-0.08860
-0.8588
-0.00319
1
3
-0.8513
0.1646
22
-5.17
<.0001
0.0001
0.05
-1.1926
-0.5100
-1.2778
-0.4249
2
3
-0.4203
0.1624
22
-2.59
0.0168
0.0504
0.05
-0.7572
-0.08348
-0.8412
0.000548
... View more
