BookmarkSubscribeRSS Feed
🔒 This topic is solved and locked. Need further help from the community? Please sign in and ask a new question.
moreka
Obsidian | Level 7

When I run a repeated measures analysis using PROC GLIMMIX, no matter which method of multiple comparison adjustment I use, the unadjusted and adjusted p-values come out the same.  

 

I can't decipher if I'm misunderstanding the statistics and/or coding, or if maybe the adjustments are so minor that they go past 4 digits and aren't displayed.

 

Example of the code:

 

proc glimmix data=all; 
class id epoch day; 
model weight = epoch|day weight0 /solution ddfm=kr;
random day/ sub=id type=cs residual;
lsmeans epoch*day / slicediff=day adjust=bon stepdown adjdfe=row;
run; 

 

Output:

              Simple Effect Comparisons of Epoch*day Least Squares Means By day
                             Adjustment for Multiple Comparisons: Holm

    Simple
    Effect                                  Standard
    Level     Epoch    Epoch    Estimate       Error       DF    t Value    Pr > |t|     Adj P

    day 1     1        2         -175.07      136.93    104.9      -1.28      0.2039    0.2039
    day 2     1        2         15.4559      136.93    104.9       0.11      0.9103    0.9103
    day 3     1        2          148.66      136.93    104.9       1.09      0.2801    0.2801
    day 4     1        2          258.16      137.61    106.6       1.88      0.0634    0.0634
    day 5     1        2          303.35      139.06    110.3       2.18      0.0313    0.0313
    day 6     1        2          426.31      146.71    130.6       2.91      0.0043    0.0043
    day 7     1        2          369.38      152.58    146.9       2.42      0.0167    0.0167

Any insights would be appreciated.

Thanks.

1 ACCEPTED SOLUTION

Accepted Solutions
sld
Rhodochrosite | Level 12 sld
Rhodochrosite | Level 12

With SLICEDIFF, the adjustment for multiple comparisons is computed within the slice (here, DAY). Documentation is your friend. For the SLICEDIFF option in the LSMEANS statement in GLIMMIX you will find:

 

"When the ADJUST= option is specified, the GLIMMIX procedure also adjusts the tests for multiplicity. The adjustment is based on the number of comparisons within each level of the SLICEDIFF= effect..."

 

EPOCH has only two levels, so there is only one pairwise comparison, and so there is no need to adjust for multiple comparisons and no effect of adjustment.

 

View solution in original post

6 REPLIES 6
ballardw
Super User

Without data or at least the procedure's other information we could well be guessing things that have no bearing on your situation.

 

If you suspect very small differences then send the results to an output data set using the ODS OUTPUT statement and then print or use other display tool with more digits in the display instead of the default p-value format.

 

If the data isn't sensitive you could use the instructions here: https://communities.sas.com/t5/SAS-Communities-Library/How-to-create-a-data-step-version-of-your-dat... to create data step code that would allow us to duplicate results and look for other issues. Subset the data to only the model variables.

 

Or create another data set without sensitive data that replicates the behavior and post that.

sld
Rhodochrosite | Level 12 sld
Rhodochrosite | Level 12

With SLICEDIFF, the adjustment for multiple comparisons is computed within the slice (here, DAY). Documentation is your friend. For the SLICEDIFF option in the LSMEANS statement in GLIMMIX you will find:

 

"When the ADJUST= option is specified, the GLIMMIX procedure also adjusts the tests for multiplicity. The adjustment is based on the number of comparisons within each level of the SLICEDIFF= effect..."

 

EPOCH has only two levels, so there is only one pairwise comparison, and so there is no need to adjust for multiple comparisons and no effect of adjustment.

 

moreka
Obsidian | Level 7

Great thanks, that makes sense.  I read through the documentation, but I figured I must be overlooking something.

So then do I understand that there is no built-in option for doing a multiplicity adjustment across the 7 days without writing specific estimate/lsmestimate statements?

sld
Rhodochrosite | Level 12 sld
Rhodochrosite | Level 12

Not that I know of.

 

I think something like this would work:

 

    lsmestimate epoch*day   "Day 1" [1, 1 1] [-1, 2 1],
                            "Day 2" [1, 1 2] [-1, 2 2],
                            "Day 3" [1, 1 3] [-1, 2 3],
                            "Day 4" [1, 1 4] [-1, 2 4],
                            "Day 5" [1, 1 5] [-1, 2 5],
                            "Day 6" [1, 1 6] [-1, 2 6],
                            "Day 7" [1, 1 7] [-1, 2 7]
    /  at means adjust=bon stepdown adjdfe=row;

You could use AT=<value> to specify your desired value of WEIGHT0.

 

Or you can capture the lsmestimates in a SAS dataset using ODS OUTPUT, do any necessary pre-processing, and then use that dataset in the MULTTEST procedure; see

 

https://support.sas.com/documentation/cdl/en/statug/63033/HTML/default/viewer.htm#statug_multtest_se...

moreka
Obsidian | Level 7
Fantastic. Thanks, that's very helpful.
SteveDenham
Jade | Level 19

This is exactly the approach we use in analyzing repeated measures in tox experiments.  Hats off to @sld.

 

Believe me, it was a bit of a shock that SLICEDIFF didn't adjust the way we naively thought it would.  And there the answer was in the documentation.  We now rely on LSMESTIMATE statements almost exclusively.

 

Steve Denham

sas-innovate-2024.png

Join us for SAS Innovate April 16-19 at the Aria in Las Vegas. Bring the team and save big with our group pricing for a limited time only.

Pre-conference courses and tutorials are filling up fast and are always a sellout. Register today to reserve your seat.

 

Register now!

What is ANOVA?

ANOVA, or Analysis Of Variance, is used to compare the averages or means of two or more populations to better understand how they differ. Watch this tutorial for more.

Find more tutorials on the SAS Users YouTube channel.

Discussion stats
  • 6 replies
  • 3332 views
  • 7 likes
  • 4 in conversation