BookmarkSubscribeRSS Feed
Fluorite | Level 6

Hello All:

   I think this is a simple question that I should know the answer to, but I just want confirmation from others.

When you are performing multiple statistical tests on a dataset and correcting for this with some type of multiple comparisons

test, such as Bonferroni or False Discovery Rate you need to know how many tests have been performed in order to perform the

proper correction. Here's a simple example with linear regression:


   Y1 = B0 + B1*X1 + B2*X2 + B3*X3


   Y2 = B0 + B1*X1 + B2*X2 + B3*X3


So there are two dependent variables, Y1 and Y2 and three predictors X1, X2, X3. It's likely that the values of B1, B2 and B3 will vary between the two equation. Perhaps you've performed 2 tests for Y1 and Y2 or (this is what I think more likely) 6 tests, estimating the values of B1 - B3 in the two contexts. Maybe even 8 tests, if you count the intercepts. I'd like opinions so that I get it right for a paper we're writing.


Obsidian | Level 7

Though there are multiple tests performed in a regression model, you need not adjust for multiple comparisons. In a regression model, there is a null model test that is generally performed to check if the null model (model with no predictors) is better than the model you've specified. If you are able to reject the null hypothesis for that test, you can proceed with interpreting coefficients that you've got without worrying about multiple comparisons problem. If you cannot reject the null, you should ignore any further interpretations you can make from the coefficients.


However, if you are performing an exploratory analyses and testing for differences for multiple different characteristics between two group-pairs using t-tests etc., you may need to adjust for multiple comparisons.


Available on demand!

Missed SAS Innovate Las Vegas? Watch all the action for free! View the keynotes, general sessions and 22 breakouts on demand.


Register now!

What is ANOVA?

ANOVA, or Analysis Of Variance, is used to compare the averages or means of two or more populations to better understand how they differ. Watch this tutorial for more.

Find more tutorials on the SAS Users YouTube channel.

Discussion stats
  • 1 reply
  • 2 in conversation