BookmarkSubscribeRSS Feed
🔒 This topic is solved and locked. Need further help from the community? Please sign in and ask a new question.
Joka
Calcite | Level 5

I have found procedures such as Tamhanes T2, Dunnets T3 and the Games & Howell procedure that deal with unequal variances in the one-way model. However, I have a Randomized complete block design, which is basically a two-way model. And variances differ quite a lot in the treatment group. In SAS for mixed models 2nd ed. (p. 369) Ramon C. Litell et. al., uses a unequal variance model,

proc mixed data=TV ic;

class age sex;

model time=sex|age/DDFM=KR OUTP=R;

repeated / group=age;

lsmeans age sex / diff adjust=Tukey;

run;

However, I am not sure this is correct since the multiple comparison test (Tukey), uses a pooled estimate for the variance thus affecting p-values when the variances are unequal. Is this the correct way of performing multiple comparison under unequal variance? If yes, how does the Tukey adjustment handle the unequal variances?

1 ACCEPTED SOLUTION

Accepted Solutions
SteveDenham
Jade | Level 19

Yes.

For your code, it would be:

proc mixed data=TV ic;

class age sex;

model time=sex|age/DDFM=KR OUTP=R;

repeated / group=age;

lsmeans age sex / diff adjust=simulate(seed=1) adjdfe=row;

run;

I picked seed=1 but any value could be inserted.  You do want to specify a seed, so that different runs are identical.

Steve Denham

View solution in original post

15 REPLIES 15
SteveDenham
Jade | Level 19

First point:

The model presented fits a separate variance for each age group, As Westfall et al. in Multiple Comparisons and Multiple Tests Using SAS, 2nd ed. say in Chapter 10, p 274 puts it "However, with extreme heteroscedasticity, Tukey's method can fail miserably."  So, I wouldn't be using method=Tukey in this case.

Second point:

Westfall et al. point out that all methods are approximate under heteroscedastic variances.  In Ch. 10, he gives several methods, and points out the pros and cons of each.  I would suggest the MaxT adjustment.

Steve Denham

Joka
Calcite | Level 5

I look this up! Is the MaxT adjustment implemented in SAS?

SteveDenham
Jade | Level 19

Yes.

For your code, it would be:

proc mixed data=TV ic;

class age sex;

model time=sex|age/DDFM=KR OUTP=R;

repeated / group=age;

lsmeans age sex / diff adjust=simulate(seed=1) adjdfe=row;

run;

I picked seed=1 but any value could be inserted.  You do want to specify a seed, so that different runs are identical.

Steve Denham

Joka
Calcite | Level 5

I am still a bit uncertain as to what kind of test is performed under heteroscedasticity and how close to the nominal levels of the presented p-values we could get. Is the method well explained in Multiple Comparisons and Multiple Tests Using SAS, 2nd ed.? I might just pick up a copy to understand this method better!

SteveDenham
Jade | Level 19

Personal opinion:  I think Westfall's book is, or should be, required reading for anyone who works with designed experiments.  There is a lot of theory mixed in, but in a way that makes it easier to understand what the code is doing.  I think the method is well explained, and there are references given that relate to the performance as well as summaries of performance.

Steve Denham

evsanten
Calcite | Level 5

Steve,

Would you recommend the sim option as a matter of SOP or are there disadvantages?

SteveDenham
Jade | Level 19

We use it as our standard method. It grew out of the ERROR message you get when Dunnett'-Hsu adjustment (previous standard) failed to converge.  SAS recommends ADJUST=SIMULATE in this case

The biggest disadvantages that we have seen are:

One-Time: an adequate number of simulations to give good results may run into long run times, especially if you are trying to meet an accuracy target, and you have a lot of endpoints.

Two-Because of the behavior of the seed and the RNG stream, it makes BY variable processing much trickier in the sense that getting identical results on separate machines is harder.  We have a good method of macro looping that avoids this problem now, however.

Steve Denham

Joka
Calcite | Level 5

Steve,

I purchased the book and I am well on my way of working through it, it was, just as you said, a really good source when working with designed experiments. Thank you!

I have a one more question that I was hoping you could answer. I am currently working through the examples in the chapter we previously discussed and I was wondering if it would be appropriate to use the maxT/minP method when analysing experiments in a randomized complete block design.

I understand that the subset pivotality condtion is depedent on the hypotheses formed to get a strong control of the FWE. However, in the case where we might suspect heteroscedasticity among group levels of treatment could the single-step maxT or minP be the an appropriate solution? Are there any pitfalls that you are aware of?

SteveDenham
Jade | Level 19

I haven't fallen into any pits yet, but I haven't had severe heteroskedasticity in any of our data.  We do see enough that, as a standard in our mixed model analyses, we model it, and accept the risk of losing power when variances are, in fact, nearly homogeneous.  The maxT, as implemented by method=sim, is our default. Hope this helps.

Steve Denham

Joka
Calcite | Level 5

Thank you for your answer!

I will try to find some simulation studies to determine power in these cases or conduct some myself!

dgbonett
Calcite | Level 5

Here is my code:

 

proc mixed;

  class group;

  model MPH = group/ddfm = satterth;

  repeated/group = group;

  lsmeans group/adjust = tukey diff cl;

 

The confidence intervals that were computed by SAS did not use Tukey critical values but used t critical values. The output indicates that the Adjustment is Tukey-Kramer so I wonder if there is a bug in the program or an error in my code.

 

 

PhilaRose
Fluorite | Level 6

Hello Steve,

I was searching the SAS forums for a solution to the same problem.  A question for you regarding your response to the original post:

Is it possible to include an interaction term in the model statement?  I am working with a data set that has two factor variables, SPECIES (three levels) and PARASITISM_STATUS (two levels). The variance in the response variable is much larger in one level of parasitism_status than the other, even when transformed. Equality of variances is not a problem for "species" when the data is log10-transformed. I am interested in the interaction between these two factor variables, so my current model statement is "model y = spp parasitism_status spp*parasitism_status."  Is there a way to include this type of interaction term in the code you provided above?

Many thanks!

SteveDenham
Jade | Level 19

The code I provided used the shorthand expression

sex|age

which is identical to:

sex age sex*age;

So the interaction was included.

For your approach, I would consider using PROC GLIMMIX, doing something like:

proc glimmix data=yourdataset;

class spp parisitism_status;

model y = spp|parasitism_status / link=log; /* Note here y is not transformed prior to analysis */

random _residual_/group=parasitism_status;

lsmeans spp parasitism_status spp*parasitism_status/exp;

lsmestimates <these will compare the lsmeans of interest to address your study objectives, and will adjust for multiple comparisons>;

run;

The lsmestimate statement is one of the finest blades in the Swiss Army knife that is GLIMMIX.  Rather than looking at all possible comparisons, as would the diff option in the lsmeans statement, you can narrow down to those of interest.  This is particularly important when looking at repeated measurements in time, where you really aren't very interested in comparing the mean of group 2 at timepoint 3 to the mean of group 5 at timepoint 11.  By eliminating comparisons such as these, the overall type I error can be controlled better without loss of power.

Steve Denham

PhilaRose
Fluorite | Level 6

Thank you so much for your reply, Steve!  And thank you for drawing my attention to the lsmestimate statment in GLIMMIX.  I'm just starting to learn how to use this platform in SAS.

How does the lsmestimate statement differ from the use of contrasts?  Currently I have a GLIMMIX procedure coded for the same dataset I described, and I've used contrast statements to examine specific comparisons. Is this problematic?

sas-innovate-2024.png

Join us for SAS Innovate April 16-19 at the Aria in Las Vegas. Bring the team and save big with our group pricing for a limited time only.

Pre-conference courses and tutorials are filling up fast and are always a sellout. Register today to reserve your seat.

 

Register now!

What is ANOVA?

ANOVA, or Analysis Of Variance, is used to compare the averages or means of two or more populations to better understand how they differ. Watch this tutorial for more.

Find more tutorials on the SAS Users YouTube channel.

Discussion stats
  • 15 replies
  • 5482 views
  • 8 likes
  • 5 in conversation