Help using Base SAS procedures

Different degree of freedom and P-value from SAS proc mixed and SPSS mixed

Reply
Super Contributor
Posts: 312

Different degree of freedom and P-value from SAS proc mixed and SPSS mixed

Hello, I am doing a repeated measure analysis using mixed model. However, SAS (9.4) and SPSS (23.0) have some difference:

1. They have different degree of freedom
2. P-values are slightly different
3. In some cases, SAS cannot converge for some covariance structure types but SPSS does not report such warning.
4. Even SAS 9.1.3 and 9.4, in some cases, have difference: sas 9.4 reports warning but sas 9.1.3 ran smoothly. Such warning is like " Convergence criteria met but final hessian is not positive definite. " Not sure why.

 

can anyone give some hints? Thank you!

 

11.png

 

 

Super User
Posts: 17,840

Re: Different degree of freedom and P-value from SAS proc mixed and SPSS mixed

[ Edited ]

At the least you'd need to post your SAS code.

Super Contributor
Posts: 312

Re: Different degree of freedom and P-value from SAS proc mixed and SPSS mixed

Here is the code I used in SAS:

 

proc mixed data=repeated.longform method=reml;
class time clientID;
model measure = time /s;
repeated  /type = arh(1) subject=ClientID r rcorr;
lsmeans time/ alpha=0.05;

estimate "Time2 vs Time1" time -1 1 0 0/cl;
estimate " Time3 vs Time1" time  -1 0 1 0/cl;
estimate " Time4 vs Time1" time  -1 0 0 1/cl;
estimate "Time3 vs Time2" time 0 -1 1 0/cl;
estimate " Time4 vs Time2" time  0 -1 0 1/cl;
estimate " Time4 vs Time3" time  0 0 -1 1/cl;

run;

 

And Here is the syntax I used for SPSS:

 

MIXED measure BY Time
  /CRITERIA=CIN(95) MXITER(100) MXSTEP(10) SCORING(1) SINGULAR(0.000000000001) HCONVERGE(0,
    ABSOLUTE) LCONVERGE(0, ABSOLUTE) PCONVERGE(0.000001, ABSOLUTE)
  /FIXED=Time | SSTYPE(3)
  /METHOD=REML
  /REPEATED=Time | SUBJECT(ClientID) COVTYPE(ARH1)
  /EMMEANS=TABLES(Time) COMPARE ADJ(LSD).

Trusted Advisor
Posts: 1,615

Re: Different degree of freedom and P-value from SAS proc mixed and SPSS mixed

I admit I don't know SPSS, but I'm not convinced you should expect to get the exact same answers. It's possible that the two different softwares use different algorithms, or possibly that they have different default options, but whatever it is they are clearly using different degrees of freedom. So if I were you, I'd try to track down the choices of algorithms, and the default options in each software.

Super Contributor
Posts: 312

Re: Different degree of freedom and P-value from SAS proc mixed and SPSS mixed

I am using SPSS is because when I chose some covariance structure SAS produce "Convergence criteria met but final hessian is not positive definite" but SPSS does not. In some cases SPSS also have the converge warning, but less than in SAS. Then I found the difference. Otherwise I may use only one software and may not notice they have difference.

Clearly they handle the degree of freedom in different ways. But they produce the same estimates which suggest the mixed model is fitted the same.

Respected Advisor
Posts: 4,651

Re: Different degree of freedom and P-value from SAS proc mixed and SPSS mixed

As a starting point on the SAS side, look at option DDFM= in proc mixed. I bet the default method (BETWITHIN for a REPEATED analysis without a RANDOM effect) isn't the same in SPSS.

PG
Super User
Posts: 10,500

Re: Different degree of freedom and P-value from SAS proc mixed and SPSS mixed

I would not be surprised to find out that different settings related to convergence or number of interations between packages you would get slightly different results.

Super Contributor
Posts: 312

Re: Different degree of freedom and P-value from SAS proc mixed and SPSS mixed

I found one answer online, for the difference in degree of freedom:

 

http://www-01.ibm.com/support/docview.wss?uid=swg21477296

 

Problem(Abstract)

I've found that a mixed linear model run in the SPSS procedure MIXED reports denominator degrees of freedom (df) for the fixed effects in the model that differ from those reported by the SAS procedure Proc Mixed. Will you explain the difference in results between the two programs?

Resolving the problem

SPSS MIXED uses the Satterthwaite approximation to calculate df. This method is valid for both balanced and unbalanced designs. By default, SAS uses the CONTAINment method when there is a RANDOM statement, and the BETWITHIN method when a REPEATED statement is used without a RANDOM statement (unless the R matrix is specified as unstructured). These methods do not adjust for lack of balance in the data. Neither does the RESIDUAL, method available in SAS. SAS does offer two methods that adjust for lack of balance: These are the SATTERTHwaite and the KENWARDROGER methods. While the CONTAIN, BETWITHIN, and RESIDUAL methods always yield integer values for the denominator df, the SATTERTHwaite and KENWARDROGER do not.

If you want to compare SAS results for mixed linear models to those from SPSS, be sure you have employed the Satterthwaite method for computing denominator degrees of freedom in SAS.

Ask a Question
Discussion stats
  • 7 replies
  • 367 views
  • 0 likes
  • 5 in conversation