topic Re: Multiple comparisons against a control in the context of an heteroscedastic ANOVA model in Statistical Procedures

Multiple comparisons against a control in the context of an heteroscedastic ANOVA model

Ben03 — Tue, 15 Dec 2015 17:51:59 GMT

Hi all,

I would like to have your input regarding a scenario where multiple pairwise comparisons against a control are conducted within the context of an Heteroscedastic Two-Way repeated measures ANOVA model.

Let's say I have 4 groups, group 1 is the control and I have multiple values taken for each group across time. Therefore, I would have an heteroscedastic ANOVA model with Group (4 levels), Time (let's say 6 levels) and their interaction as fixed factors.

The pairwise comparisons of interest would be Group 1 vs Group 2, Group 1 vs Group 3 and Group 1 vs Group 4.

My SAS mixed procedure would be as follows:

PROC MIXED Data= data METHOD=REML ;

CLASS Group Time Subject ;

MODEL Value = Group Time Group*Time / DDFM=KR ;

REPEATED Time / GRP=Group Subject=Subject TYPE=CS ;/* this could be a different covariance structure */

LSMEANS Group / PDIFF=Control("1") ADJUST= ??? ADJDFE=??? ;

RUN ;

I would greatly appreciate to know your input for the best approach to consider regarding the "???" that I have included in the above SAS code.

Thanks

Re: Multiple comparisons against a control in the context of an heteroscedastic ANOVA model

lvm — Thu, 17 Dec 2015 14:16:38 GMT

Your code is OK. There are many choices for adjustments. I recommend adj=simulate. This works well for most problems, and gets around the assumptions underlying the other adjustment methods. For your application, you probably don't need a df adjustment. However, with the KR adjustment in the model statement, you could use adjdfe=row to take full advantage of the different df you may have for different means.

With this repeated measures, you probably also want to look into slices -- comparisons of means for one factor at each level of the 'other' factor.

Re: Multiple comparisons against a control in the context of an heteroscedastic ANOVA model

SteveDenham — Wed, 30 Dec 2015 19:21:49 GMT

I support @lvm's response here. Your current LSMEANS statement compares differences in treatments averaged over all time points, and I am willing to wager upwards of 5 dollars US that you really want to compare differences in treatments at EACH time point. That is what the SLICE and SLICEDIFF option are for.

Steve Denham

Re: Multiple comparisons against a control in the context of an heteroscedastic ANOVA model

Ben03 — Wed, 13 Jan 2016 13:42:38 GMT

Hi @SteveDenham and @lvm,

Thank you both for your response. Please note that I had voluntarely remove the LSMEANS statement with the SLICE option in order to put the focus on the pairwise comparion's test performed across all timepoints.

Here is my full code:

PROC MIXED DATA=data METHOD=REML ;

CLASS Group Time Subject ;

MODEL Value = Group Time Group*Time / DDFM=KR ;

REPEATED Time / GRP=Group Subject=Subject TYPE=CS ; /*this could be a different covariance structure */

LSMEANS Group / PDIFF=Control("1") ADJUST=??? ADJDFE=??? ;

LSMEANS Group*Time / PDIFF SLICE=Time ;

RUN ;

My main concern was regarding if the Dunnett test could be used in the context of an Heteroscedastic ANOVA model in order to compare all treated groups to a control group across all timepoints using the following statement:

LSMEANS Group / PDIFF=Control("1") ADJUST=Dunnett ADJDFE=ROW ;

Thanks

Re: Multiple comparisons against a control in the context of an heteroscedastic ANOVA model

SteveDenham — Wed, 13 Jan 2016 13:52:57 GMT

Well, it CAN, but that doesn't mean it is the best option. ADJUST=simulate is by far the best option available in SAS for repeated measures analyses.

Steve Denham

Re: Multiple comparisons against a control in the context of an heteroscedastic ANOVA model

Ben03 — Wed, 13 Jan 2016 13:57:36 GMT

Thank you Steve. Would you have any kind of documentation or paper that you could please refer to me regarding this adjustment?

Thanks

Re: Multiple comparisons against a control in the context of an heteroscedastic ANOVA model

Rick_SAS — Wed, 13 Jan 2016 14:14:58 GMT

The advantage of the SIMULATE option is that it does not require as many assumptions about the data, and I suspect that is whySteve asserted that it is "the best." I suspect that he meant that in practice simulation is very flexible and works for a wide range of data and models. If the data truly is a random sample from some distribution that is assumed by one of the other methods, then that method might give better results FOR THAT SAMPLE.

Re: Multiple comparisons against a control in the context of an heteroscedastic ANOVA model

lvm — Wed, 13 Jan 2016 14:15:39 GMT

I would cite P.H. Westfall, R.D. Tobias, and R.D. Wolfinger. 2011. Multiple Comparisons and Multiple Tests Using SAS, second edition. SAS Press.

The adjust=simulate is my favorite method.

Re: Multiple comparisons against a control in the context of an heteroscedastic ANOVA model

Rick_SAS — Wed, 13 Jan 2016 14:24:20 GMT

And if you use ADJUST=SIMULATE(REPORT) , you can see useful information about the estimation.