02-13-2017 04:11 AM
I've been evaluating SAS 9.4M3 versus 9.4M1 and I've come across a few differences in the output for certain procedures.
Have others also encountered inconsistent results from statistical models using SAS 9.4M3 versus previous versions of SAS? And can anyone provide insights on when PROC GLIMMIX might produce different results in the two versions?
02-13-2017 09:09 AM
See these previous threads that asked similar questions:
02-13-2017 10:43 AM
Thanks for the links, it's interesting to see that similar problems have cropped up before. In these two cases, the problem seems to be due to a message that the "estimated G-matrix was not positive definite". To me, that means that the covariance structure of the model needs to be simplified.
I've looked up one of the GLIMMIX models that showed differences. This was a logistic regression model with a random intercept with two third order interaction effects, with a total of 24 fixed coefficients. The program converged after 112 iterations under SAS 9.4M1 versus 101 iterations under 9.4M3. The final -2 log likelihood was slightly better in 9.4M1 (583.69231144 versus 583.72405139). The fixed effects were similar but with differences in the first decimal place.
I did notice a note in the SAS log:
NOTE: At least one element of the gradient is greater than 1e-3.
In Tips and Strategies for Mixed Modeling with SAS/STAT® Procedures (page 13), I found this message is usually not a problem but can be dealt with by adding the following statement to proc glimmix:
When I added that, SAS 9.4M1 required 147 iterations and the convergence criterion changed to FCONV=2.220446E-16. SAS 9.4M3 needed 170 iterations but produced the same -2 LL value and the same model coefficients (2 fixed effects had differences of +/- 0.0001).
So I think the upshot is that you have to be extra careful using PROC GLIMMIX if you get log messages. And that SAS has changed something in the convergence criteria without properly documenting it.