When I run a repeated measures analysis using PROC GLIMMIX, no matter which method of multiple comparison adjustment I use, the unadjusted and adjusted p-values come out the same.
I can't decipher if I'm misunderstanding the statistics and/or coding, or if maybe the adjustments are so minor that they go past 4 digits and aren't displayed.
Example of the code:
proc glimmix data=all;
class id epoch day;
model weight = epoch|day weight0 /solution ddfm=kr;
random day/ sub=id type=cs residual;
lsmeans epoch*day / slicediff=day adjust=bon stepdown adjdfe=row;
run;
Output:
Simple Effect Comparisons of Epoch*day Least Squares Means By day Adjustment for Multiple Comparisons: Holm Simple Effect Standard Level Epoch Epoch Estimate Error DF t Value Pr > |t| Adj P day 1 1 2 -175.07 136.93 104.9 -1.28 0.2039 0.2039 day 2 1 2 15.4559 136.93 104.9 0.11 0.9103 0.9103 day 3 1 2 148.66 136.93 104.9 1.09 0.2801 0.2801 day 4 1 2 258.16 137.61 106.6 1.88 0.0634 0.0634 day 5 1 2 303.35 139.06 110.3 2.18 0.0313 0.0313 day 6 1 2 426.31 146.71 130.6 2.91 0.0043 0.0043 day 7 1 2 369.38 152.58 146.9 2.42 0.0167 0.0167
Any insights would be appreciated.
Thanks.
With SLICEDIFF, the adjustment for multiple comparisons is computed within the slice (here, DAY). Documentation is your friend. For the SLICEDIFF option in the LSMEANS statement in GLIMMIX you will find:
"When the ADJUST= option is specified, the GLIMMIX procedure also adjusts the tests for multiplicity. The adjustment is based on the number of comparisons within each level of the SLICEDIFF= effect..."
EPOCH has only two levels, so there is only one pairwise comparison, and so there is no need to adjust for multiple comparisons and no effect of adjustment.
Without data or at least the procedure's other information we could well be guessing things that have no bearing on your situation.
If you suspect very small differences then send the results to an output data set using the ODS OUTPUT statement and then print or use other display tool with more digits in the display instead of the default p-value format.
If the data isn't sensitive you could use the instructions here: https://communities.sas.com/t5/SAS-Communities-Library/How-to-create-a-data-step-version-of-your-dat... to create data step code that would allow us to duplicate results and look for other issues. Subset the data to only the model variables.
Or create another data set without sensitive data that replicates the behavior and post that.
With SLICEDIFF, the adjustment for multiple comparisons is computed within the slice (here, DAY). Documentation is your friend. For the SLICEDIFF option in the LSMEANS statement in GLIMMIX you will find:
"When the ADJUST= option is specified, the GLIMMIX procedure also adjusts the tests for multiplicity. The adjustment is based on the number of comparisons within each level of the SLICEDIFF= effect..."
EPOCH has only two levels, so there is only one pairwise comparison, and so there is no need to adjust for multiple comparisons and no effect of adjustment.
Great thanks, that makes sense. I read through the documentation, but I figured I must be overlooking something.
So then do I understand that there is no built-in option for doing a multiplicity adjustment across the 7 days without writing specific estimate/lsmestimate statements?
Not that I know of.
I think something like this would work:
lsmestimate epoch*day "Day 1" [1, 1 1] [-1, 2 1], "Day 2" [1, 1 2] [-1, 2 2], "Day 3" [1, 1 3] [-1, 2 3], "Day 4" [1, 1 4] [-1, 2 4], "Day 5" [1, 1 5] [-1, 2 5], "Day 6" [1, 1 6] [-1, 2 6], "Day 7" [1, 1 7] [-1, 2 7] / at means adjust=bon stepdown adjdfe=row;
You could use AT=<value> to specify your desired value of WEIGHT0.
Or you can capture the lsmestimates in a SAS dataset using ODS OUTPUT, do any necessary pre-processing, and then use that dataset in the MULTTEST procedure; see
This is exactly the approach we use in analyzing repeated measures in tox experiments. Hats off to @sld.
Believe me, it was a bit of a shock that SLICEDIFF didn't adjust the way we naively thought it would. And there the answer was in the documentation. We now rely on LSMESTIMATE statements almost exclusively.
Steve Denham
Join us for SAS Innovate April 16-19 at the Aria in Las Vegas. Bring the team and save big with our group pricing for a limited time only.
Pre-conference courses and tutorials are filling up fast and are always a sellout. Register today to reserve your seat.
ANOVA, or Analysis Of Variance, is used to compare the averages or means of two or more populations to better understand how they differ. Watch this tutorial for more.
Find more tutorials on the SAS Users YouTube channel.