Different output in SAS 9.4M3 (SAS/STAT 14.1)?

[ckx]

I've been evaluating SAS 9.4M3 versus 9.4M1 and I've come across a few differences in the output for certain procedures.

• PROC MIXED can sometimes produce a different number of numerator df for fixed effects (solution and type 3 tests). Oddly, all other results are identical - estimates, S.E., p-values. This doesn't always happen, but I've come across 1 instance at least.
• PROC LOGISTIC can produce a warning on "quasi-complete separation of data points" when dealing with very sparse data in a cumulative odds model. For the cases I've examined, this is a solid improvement. This occurred with tabular data from adverse events analysis of e.g. the effects of treatment group on severity. The tables were often very sparse, with counts < 5 and sometimes a count of 0. SAS 9.4M1 would produce a p-value of dubious value with no warning.
• PROC GLIMMIX could also produce different results. This is something I have no first-hand experience with and I'm not an expert at PROC GLIMMIX models. Apparently, models could require a different number of iterations and produce different results using SAS 9.4M1 versus 9.4M3.

Have others also encountered inconsistent results from statistical models using SAS 9.4M3 versus previous versions of SAS? And can anyone provide insights on when PROC GLIMMIX might produce different results in the two versions?

[Ksharp]

You also need to consider random effect is G-side or R-side.

[Rick_SAS]

See these previous threads that asked similar questions:

Differences in arh(1) between SAS 9.2 and 9.3

Consistency of GLIMMIX results between 9.3 and 9.4 versions

[ckx]

Hi Rick,

Thanks for the links, it's interesting to see that similar problems have cropped up before. In these two cases, the problem seems to be due to a message that the "estimated G-matrix was not positive definite". To me, that means that the covariance structure of the model needs to be simplified.

I've looked up one of the GLIMMIX models that showed differences. This was a logistic regression model with a random intercept with two third order interaction effects, with a total of 24 fixed coefficients. The program converged after 112 iterations under SAS 9.4M1 versus 101 iterations under 9.4M3. The final -2 log likelihood was slightly better in 9.4M1 (583.69231144 versus 583.72405139). The fixed effects were similar but with differences in the first decimal place.

I did notice a note in the SAS log:

NOTE: At least one element of the gradient is greater than 1e-3.

In Tips and Strategies for Mixed Modeling with SAS/STAT® Procedures (page 13), I found this message is usually not a problem but can be dealt with by adding the following statement to proc glimmix:

nloptions gconv=0;

When I added that, SAS 9.4M1 required 147 iterations and the convergence criterion changed to FCONV=2.220446E-16. SAS 9.4M3 needed 170 iterations but produced the same -2 LL value and the same model coefficients (2 fixed effects had differences of +/- 0.0001).

So I think the upshot is that you have to be extra careful using PROC GLIMMIX if you get log messages. And that SAS has changed something in the convergence criteria without properly documenting it.