topic lsmeans "adjust=" not working in Statistical Procedures

lsmeans "adjust=" not working

matthew_m — Tue, 17 Jul 2018 17:02:26 GMT

I'm running the following code, and running into an issue with the adjustment for multiple comparisons:

proc glimmix data=Weight;
	class ID Sex Virus Day;
	where sex=1;
	WtChange = Day|Virus;
	random Day / residual subject=ID type=ar(1);
	lsmeans Day*Virus / slicediff=day slice=virus plot=mean(sliceby=Virus join clband) adjust=bon;
	ods output slicediffs=Female_BL_Wt_PH;
run;

Everything seems to be working well, with the exception of the correction for multiple comparisons (adj_p = raw_p). No adjustment appears to be made. I included a picture of the table below to illustrate my output:

I tried adjust=bon, tukey, simulate, and none seem to work. Any thoughts on what is going on? My fixed effect appears to be significant, and there are clearly interesting differences in my lsmeans diffs (see table above). Could this have to do with the slice function? Thanks for the help!

Re: lsmeans "adjust=" not working

ballardw — Tue, 17 Jul 2018 17:21:04 GMT

The displayed table may not show actual differences if they are small.

Try printing your output set with a wider format than the default pvalue format such as Best12. and see if they match to 10 decimals.

Re: lsmeans "adjust=" not working

matthew_m — Tue, 17 Jul 2018 17:34:54 GMT

Interesting thought, but that didn't seem to fix the issue... They're still identical. Plus, I'd be surprised if the bonferroni correction didn't affect my results. Table with expanded p value below.

Re: lsmeans "adjust=" not working

sld — Wed, 18 Jul 2018 04:17:27 GMT

The pairwise comparisons in your table are generated by "slicediff=day". There are only two levels of virus, so there is only one comparison for each day. The adjustment (regardless of which method) is made within each slice. When there is only one comparison, there is no adjustment.

Quoting the documentation found here:

"When the ADJUST= option is specified, the GLIMMIX procedure also adjusts the tests for multiplicity. The adjustment is based on the number of comparisons within each level of the SLICEDIFF= effect; see the SLICEDIFFTYPE= option. "

(Documentation is your friend and provides many answers.)

Re: lsmeans "adjust=" not working

matthew_m — Wed, 18 Jul 2018 13:19:23 GMT

Hmmm... That not the way I interpreted that statement. If I understand the way you're interpreting it, the adjust statement is only correcting for multiple comparisons within each level of "Day". In other words, since I'm making 1 comparison on each day, it is only correcting for 1 comparison. It is not correcting for the 9 comparisons listed in the table, which actually what I want it to do. Is there a better way to code this within the lsmeans statement, or will I be forced to export the table and use proc multtest? Thanks for all the help.

Re: lsmeans "adjust=" not working

sld — Wed, 18 Jul 2018 13:53:59 GMT

I would say that your interpretation of my interpretation is correct. And consistent with the results you get.

An argument could be made that there is little need for Type I error control among these 9 comparisons: no mean is used in more than one comparison, and the overall test of interaction assesses whether all 9 differences are equal (versus the alternative that at least two differences are unequal).

If you wanted control then, as you note, you could save the SLICEDIFF output to a dataset using ODS and then use MULTTEST. You could also specify the 9 comparisons in a LSMESTIMATE statement with the ADJUST option.