Programming the statistical procedures from SAS

What adjustment to pick in a GLIMMIX variance analysis

Reply
New Contributor
Posts: 3

What adjustment to pick in a GLIMMIX variance analysis

I am using Glimmix to do a variance analysis of 96 treatments, each with 3 blocks (reps). My question is whether tukey would by the correct adjustment to use as this would give 4560 pairwise combos, my lsmeans is shown below and my treatment is "strain"

 

lsmeans strain/pdiff adjust=tukey; 

 

The experiment is a screen to find the highest NH4 excretion, and I am wondering if another adjustment would make more sense with this number of treatments

PROC Star
PROC Star
Posts: 404

Re: What adjustment to pick in a GLIMMIX variance analysis

Speculating here....

 

To me, screening is exploratory. If the objective of this study is to select a smaller number of treatments for future study, I could see doing no adjustment at all. (I'm sure someone somewhere would disagree.)

 

Or you could explicitly balance the costs of Type I and Type II errors; for example, see Setting an Optimal α That Minimizes Errors in Null Hypothesis Significance Tests; but see LOPSIDED REASONING ON LOPSIDED TESTS AND MULTIPLE COMPARISONS for an opposing opinion. 

 

Of the usual Type I error control methods, I might consider false discovery rate rather than something like Tukey.

 

I doubt that there is a definitive answer to your question, although there may be something like a consensus in your particular discipline (which might not be the same consensus in another discipline).

 

 

New Contributor
Posts: 3

Re: What adjustment to pick in a GLIMMIX variance analysis

Thanks for the well thought out reply, I think Tukey increases my type two error rate to a large margin, and having no adjustment changes this.

 

I am going to inquire with others in my discipline on what they think of having no adjustment for this screen.

 

Thanks for the advice.

 

All the best, Chris

PROC Star
PROC Star
Posts: 404

Re: What adjustment to pick in a GLIMMIX variance analysis

Using simulation studies, Day and Quinn (1989) COMPARISONS OF TREATMENTS AFTER AN ANALYSIS OF VARIANCE IN ECOLOGY showed that the Tukey method is conservative with respect to Type I error rate; in Fig 1 you can see that the actual Type I error rate is about 0.02 when the nominal rate is 0.05 for their simulation scenario. Consequently, Type II error rate also is excessively increased (i.e., you lose more power). There's no perfect solution Smiley Happy

 

Ask a Question
Discussion stats
  • 3 replies
  • 141 views
  • 0 likes
  • 2 in conversation