BookmarkSubscribeRSS Feed
cdumigan
Calcite | Level 5

I am using Glimmix to do a variance analysis of 96 treatments, each with 3 blocks (reps). My question is whether tukey would by the correct adjustment to use as this would give 4560 pairwise combos, my lsmeans is shown below and my treatment is "strain"

 

lsmeans strain/pdiff adjust=tukey; 

 

The experiment is a screen to find the highest NH4 excretion, and I am wondering if another adjustment would make more sense with this number of treatments

3 REPLIES 3
sld
Rhodochrosite | Level 12 sld
Rhodochrosite | Level 12

Speculating here....

 

To me, screening is exploratory. If the objective of this study is to select a smaller number of treatments for future study, I could see doing no adjustment at all. (I'm sure someone somewhere would disagree.)

 

Or you could explicitly balance the costs of Type I and Type II errors; for example, see Setting an Optimal α That Minimizes Errors in Null Hypothesis Significance Tests; but see LOPSIDED REASONING ON LOPSIDED TESTS AND MULTIPLE COMPARISONS for an opposing opinion. 

 

Of the usual Type I error control methods, I might consider false discovery rate rather than something like Tukey.

 

I doubt that there is a definitive answer to your question, although there may be something like a consensus in your particular discipline (which might not be the same consensus in another discipline).

 

 

cdumigan
Calcite | Level 5

Thanks for the well thought out reply, I think Tukey increases my type two error rate to a large margin, and having no adjustment changes this.

 

I am going to inquire with others in my discipline on what they think of having no adjustment for this screen.

 

Thanks for the advice.

 

All the best, Chris

sld
Rhodochrosite | Level 12 sld
Rhodochrosite | Level 12

Using simulation studies, Day and Quinn (1989) COMPARISONS OF TREATMENTS AFTER AN ANALYSIS OF VARIANCE IN ECOLOGY showed that the Tukey method is conservative with respect to Type I error rate; in Fig 1 you can see that the actual Type I error rate is about 0.02 when the nominal rate is 0.05 for their simulation scenario. Consequently, Type II error rate also is excessively increased (i.e., you lose more power). There's no perfect solution 🙂

 

sas-innovate-2024.png

Join us for SAS Innovate April 16-19 at the Aria in Las Vegas. Bring the team and save big with our group pricing for a limited time only.

Pre-conference courses and tutorials are filling up fast and are always a sellout. Register today to reserve your seat.

 

Register now!

What is ANOVA?

ANOVA, or Analysis Of Variance, is used to compare the averages or means of two or more populations to better understand how they differ. Watch this tutorial for more.

Find more tutorials on the SAS Users YouTube channel.

Discussion stats
  • 3 replies
  • 1229 views
  • 0 likes
  • 2 in conversation