Programming the statistical procedures from SAS

Multiple comparison for only treatment effects

Accepted Solution Solved
Reply
New Contributor
Posts: 2
Accepted Solution

Multiple comparison for only treatment effects

Hey all

I have a question which I cannot seem to solve.

I am working with brain imaging data. I have two "treatment" conditions (before and after) as well as several brain "regions" (n=18). In each region I would like to examine the effect from my treatment. The test is a repeated meassure design where the same "subjects" undergo both treatment and non-treatment condition.

First I am interested in the two way ANOVA for "condition" "Region" and "condition*Region". This seems to be working just great!

However secondly I want to do post-hoc testing to see which brain regions are significantly affected by my treatment. am therefore only interested in how my treatment affects each indivual brain region (the interaction between "treatment*region") not if different brain regions differ from one another!

In the code below SAS only provides me with output of the "treatment*region" interaction. However as far as I can tell SAS still has done all the other multiple comparisons between brain regions. Since I have 18 brain regions this means that I end up with no significant results..

Here is my code:

PROC sort data=DataPet;

by Region subjects Condition;

proc print data=DataPet;

run;

proc mixed data=DataPet;

class subjects Region condition;

model Activity=condition Region condition*Region;

lsmeans Region*condition/diff adjust=Bon; 

ods output diffs=temp; Repeated / Subject=subjects;

run;

data DataPet2;

set temp;

if Region=_Region;

proc print data=DataPet2;

var Region _Region condition _condition estimate stderr df tvalue probt;

run;

I would be very happy if anyone could tell me if I am doing the multiple comparison correctly. And if not, how I can modify my code to get the post-hoc output I am interested in, namely only the treatment*region interaction.

Thank you very much in advance!


Accepted Solutions
Solution
‎08-03-2011 09:39 AM
Valued Guide
Valued Guide
Posts: 684

Multiple comparison for only treatment effects

It is not clear what you are asking. If you look at the MIXED output, you will see the pairwise comparison of all region*condition combinations (2*18=36 interaction means, which would give 36*35/2 = 630 pairwise comparisons). Your post-model fitting code is then eliminating all the pairs where the region is not the same. In other words, it looks like you will then print a table with 18 brain regions, and the estimate is the pairwise difference of the two treatment lsmeans. Is that what you want? Seems like a good plan to me. All the results in file DATAPet2 are also (buried) in the full listing of pairwise mean differences in the MIXED output. Note: this only makes sense if you have an interaction.

With so many differences, the Bonferonni adjustment (Bon) method will be very conservative. In other words, it is probably getting extremely difficult to find any significant differences, even with a highly significant global test result. I highly recommend that you use a different adjustment. Try adjust=tukey; or adjust=simulate; . The latter is my favorite, but there are advocates of different adjusment methods.

If you have a global interaction, you might want to get slices (simple main effects) for each region. That is, you can easily get MIXED to tell you if treatment has an effect for each region in a nice table. The lsmeans statement would then become:

lsmeans region*condition/ diff adjust=simulate slice=region;

I also think that your repeated statement is not right. Based on what you wrote, both condition and region are repeated measures. You are getting a simple variance-component model, and at a minimum, you need to account for variability between and within subjects. One simple approach is to use type=CS as an additional option on your repeated statement. But, you probably want to explore more complex structures. For instance, there is a spatial orientation to region, so you could model this in the repeated statement, and there are several choices. If you pursued this later idea, there are numerous issues to consider, and it would require that you read about the subject of spatial repeated measures. You could start with the the excelellent book by Littell et al. (SAS for Mixed Models, 2nd edition). The book by Brown and Prescott on mixed models in medicine is also excellent (and SAS based).

View solution in original post


All Replies
Solution
‎08-03-2011 09:39 AM
Valued Guide
Valued Guide
Posts: 684

Multiple comparison for only treatment effects

It is not clear what you are asking. If you look at the MIXED output, you will see the pairwise comparison of all region*condition combinations (2*18=36 interaction means, which would give 36*35/2 = 630 pairwise comparisons). Your post-model fitting code is then eliminating all the pairs where the region is not the same. In other words, it looks like you will then print a table with 18 brain regions, and the estimate is the pairwise difference of the two treatment lsmeans. Is that what you want? Seems like a good plan to me. All the results in file DATAPet2 are also (buried) in the full listing of pairwise mean differences in the MIXED output. Note: this only makes sense if you have an interaction.

With so many differences, the Bonferonni adjustment (Bon) method will be very conservative. In other words, it is probably getting extremely difficult to find any significant differences, even with a highly significant global test result. I highly recommend that you use a different adjustment. Try adjust=tukey; or adjust=simulate; . The latter is my favorite, but there are advocates of different adjusment methods.

If you have a global interaction, you might want to get slices (simple main effects) for each region. That is, you can easily get MIXED to tell you if treatment has an effect for each region in a nice table. The lsmeans statement would then become:

lsmeans region*condition/ diff adjust=simulate slice=region;

I also think that your repeated statement is not right. Based on what you wrote, both condition and region are repeated measures. You are getting a simple variance-component model, and at a minimum, you need to account for variability between and within subjects. One simple approach is to use type=CS as an additional option on your repeated statement. But, you probably want to explore more complex structures. For instance, there is a spatial orientation to region, so you could model this in the repeated statement, and there are several choices. If you pursued this later idea, there are numerous issues to consider, and it would require that you read about the subject of spatial repeated measures. You could start with the the excelellent book by Littell et al. (SAS for Mixed Models, 2nd edition). The book by Brown and Prescott on mixed models in medicine is also excellent (and SAS based).

New Contributor
Posts: 2

Multiple comparison for only treatment effects

Thanks a lot for the answer, that was very helpfull!

I looked into the Compound Symetry comand as you suggested and I agree that I indeed need to add that to my code. My results now look much more like what I had expected from the manually bonferoni corrected t-tests.

However before I celebrate I still have one concern. Changing the post-hoc adjustment method from adjust=tukey to adjust=Bon or to adjust=simulate does not change the test outcome. Does that make sence to you because this I do not really understand?

And thank you for the book suggestions as well, since I will most certainly be conducting similar tests in the future I will go get me a copy of thed book by Littell :-)

Valued Guide
Valued Guide
Posts: 684

Multiple comparison for only treatment effects

You normally should see a big effect of using Bonferroni vs. Tukey (etc.), unless the overall effect was not significant. The Bon. adjustment would be huge in your case, with so many comparisons. It would reguire extremely large t values (or extremely small [unadjusted] p values) for a pairwise difference to be significant. I am a bit suspicious that you are not seeing a big effect.

🔒 This topic is solved and locked.

Need further help from the community? Please ask a new question.

Discussion stats
  • 3 replies
  • 211 views
  • 0 likes
  • 2 in conversation