topic Should I report type III fixed effects stats and a priori contrasts stats, or contrast stats in Statistical Procedures

Should I report type III fixed effects stats and a priori contrasts stats, or contrast stats

didymo — Tue, 17 May 2022 22:04:39 GMT

I have a repeated measures ANOVA Y=treatment year treatment*year but I am interested in two specific a priori contrasts. Should I report the main effect treatment*year stats along with the a priori contrast stats or just the a priori contrast stats? Thanks, BT

Re: Should I report type III fixed effects stats and a priori contrasts stats, or contrast stats

SteveDenham — Wed, 18 May 2022 12:21:02 GMT

Provided the a priori hypotheses are clearly stated, the only p values you need are those for the comparisons. The stagewise procedure of looking at the Type III p value as a gateway to post hoc comparisons isn't needed.

SteveDenham

PS. A Bayesian analysis would make this more apparent.

Re: Should I report type III fixed effects stats and a priori contrasts stats, or contrast stats

didymo — Wed, 18 May 2022 13:22:39 GMT

Thank you Steve. However, in some cases if I explore Tukey's postdoc tests, then I would report type III p values of treatment year, and treatment*year as one would normally do, right?

My knowledge of Bayesian stats is limited unfortunately.

Re: Should I report type III fixed effects stats and a priori contrasts stats, or contrast stats

SteveDenham — Thu, 19 May 2022 12:21:08 GMT

I agree. If you do not have pre-planned comparisons, some form of adjustment for multiplicity should be done to control Type I error.

One of the advantages of Bayesian approaches is that since the results do not have distributional or hypothesis assumptions and you can specify your utility (misclassification), you probably shouldn't be doing adjustments for multiplicity (per F. Harrell).

SteveDenham