Different results for mixed ANOVA in SAS and SPSS

[obrienfk]

Hi all,

I'm conducting a mixed ANOVA. I have an independent groups variable (Group) with 2 levels and a repeated measures variable (Time) with 4 levels. I want to see if there's a main effect of Group, a main effect of Time and/or an interaction between the two.

In SPSS, this is simply done through the main ANOVA options. Because Group has only two levels, the analysis for the main effect of this variable is the same as doing a simple t-test. Indeed, in SPSS the exact same results come about for the Group part of the ANOVA as they do for a separate t-test (they both reveal Group to be non-significant).

In SAS, I have been using the PROC MIXED statement to conduct the a mixed ANOVA with this exact code (for a dataset in Long Format):

PROC MIXED DATA = MyData;

CLASS Time Group IDSubject;

  MODEL Score = Time Group Group*Time;

  REPEATED Time/ SUBJECT = IDSubject;

RUN;

What's interesting is that the when I run this code, group is found to be significant! When I run a t-test using SAS, the results (non-significant) are the same as the t-test in SPSS and the main effect analysis for group in the Mixed ANOVA in SPSS as well.

My question is, why am I getting a different result in the Mixed ANOVA in SAS compared with SPSS. I'm assuming it has something to do with different calculations since the PROC mixed seems to be used for other things like Multilevel modelling.

I'd appreciate any help.

[SteveDenham · Accepted solution]

I am not at all familiar with SPSS, but the difference between the t test and PROC MIXED is that the t test assumes the observations are independent while the mixed model accommodates correlation over time. By removing this source of variability, greater precision is obtained for the estimates and consequently significance is "found".

My suspicion about the mixed ANOVA in SPSS is that it is more like PROC GLM and how it accommodates random effects, partitioning sums of squares, rather than finding a (restricted) maximum likelihood solution.  Getting results similar to the t test approach would tend to support this, I think.

As far as the analysis in PROC MIXED, consider adding a type= option to the REPEATED statement that reflects the covariance structure of the data.  If the data are equally spaced in time, autoregressive structures are likely to fit the data better than the default variance components structure.  If not equally spaced in time, structures such as spatial power  are often appropriate.

Steve Denham

[Doc_Duke]

Second Steve's comment on GLM.  If you did the REPEATED statement in GLM (rather than MIXED), I'd suspect that you would get nearly identical results to SPSS.

[obrienfk]

Thank you both for your replies. Doc@Duke, I didn't actually realize that GLM in SAS could do repeated measures - that was the whole reason I was using mixed.

Steve, is it generally better to use this MIXED (because it "accommodates correlation over time") compared with GLM -or are there circumstances where a repeated measures (or mixed method) ANOVA should be carried out in PROC GLM?

I don't know much about "covariance structures of the data" -I'll have to look into it.

[SteveDenham]

Personal preference--I would never do a repeated measures in time (longitudinal design) in GLM, as the restrictions on the covariance structure are not reflective of physical, biological or economical processes.  The sphericity assumption and the work-around Huynh-Feldt or Greenhouse-Geisser adjustments all assume an "exchangeable" structure--that the actual ordering of the time variables makes no difference in the error.  Advances in methodology mean that we do not have to make such simplifying assumptions.  For similar reasons, I would not use GLM for random effects analyses.

I use GLM for regression when there are class variables, and for multivariate ANOVA.  Otherwise, I use GLIMMIX--it handles almost everything available in MIXED, plus it can deal with non-normally distributed endpoints.

Steve Denham