I'm running tests on a dataset where my CLASS variable has 9 levels. Two of the levels are very similar, and I want to determine whether they are significantly different from each other to see if they can actually be separated or whether they need to be combined. When I run PROC TTEST restricted to these two levels, they are shown to be significantly different from each other (Pr > |t| is <0.0001) and (Pr > F is <0.0001). When I run a PROC GLM modeling the same 'var' variable for these two levels, however, the CONTRAST statement returns a significant yet different result (Pr > F = 0.0407). Should I expect TTEST and CONTRAST to find the same significance? Is it recommended to rely on one or the other when testing for a significant difference? Also, if I run the full model (with an additional CLASS variable), the Pr > F in the CONTRAST output for the two levels increases to 0.3231. Would you think that the levels need to be significantly different from each other in this full model, or only in the more basic model? I realize that perhaps this decision may be left to the discretion of the modeler.
... View more