PROC TTEST versus PROC GLM - CONTRAST

[AJones]

I'm running tests on a dataset where my CLASS variable has 9 levels.  Two of the levels are very similar, and I want to determine whether they are significantly different from each other to see if they can actually be separated or whether they need to be combined.

When I run PROC TTEST restricted to these two levels, they are shown to be significantly different from each other (Pr > |t|  is  <0.0001) and (Pr > F is <0.0001).

When I run a PROC GLM modeling the same 'var' variable for these two levels, however, the CONTRAST statement returns a significant yet different result (Pr > F = 0.0407). 

Should I expect TTEST and CONTRAST to find the same significance?  Is it recommended to rely on one or the other when testing for a significant difference?

Also, if I run the full model (with an additional CLASS variable), the Pr > F in the CONTRAST output for the two levels increases to 0.3231.  Would you think that the levels need to be significantly different from each other in this full model, or only in the more basic model?  I realize that perhaps this decision may be left to the discretion of the modeler.

[PaigeMiller · Accepted solution]

If you create a PROC TTEST analysis using two levels, you need to compare that to PROC GLM restricted to tthe same two levels. The probability of rejecting the null hypothesis should then match across the two procedures.

If you compare PROC TTEST based on two levels and a PROC GLM contrast of two levels when the entire PROC GLM analysis is based on 9 levels, they won't match.

So which should you use? If all you care about is the two levels and not the other  seven levels, then use PROC TTEST. If the variability across and within all 9 levels is of interest, then you would use PROC GLM.

[AJones]

Thank you.  I thought I was restricting to the two levels by using a WHERE statement, but in fact I had not thought to delete the 7 extra zero placeholders in the CONTRAST statement.  That must have been throwing it off, because the probability does indeed now match the TTEST. 

I appreciate your response!