Convergence problem with glimmix for a random-intercept log linear model

[Demographer]

Hi,

I'm trying since days to make what should be a quite simple model but for some reasons, I can't make them converge.

I want to estimate the impact of the number of years of education (nbedu) on salary, controlling for age and sex. My dataset combine many surveys from different countries and different years (36 surveys in total on 14 countries). I also want an interaction with the variable "edugr" which is the average number of years of education in the country, to see if the impact of the education on salary is in countries with low educational attainment.

I was thus thinking to make a log linear model with random intercepts for the survey (identified by the variable "sample), and possibly random slope for the number of years of education. Since the variable "salary" has very different values among survey, I normalized it to  sample average=1 (norm_sal). The total sample size is higher than 3M.

I tried different models with different specification (such as adding random effect for nbedu and edugr, using the non-normalized salary, adding interaction between age and sex). I also tried to increase the number of iteration and other options, but nothing work. This is just one of them.

proc glimmix data=test.smallsample INITGLM;
class edugr SEXR sample ;
model norm_sal= nbedu|edugr agegr SEXR / solution dist=gamma link=log;
random intercept / subject=sample;
weight perwt;
NLOPTIONS MAXITER=200 ;
run;

A random sample of the dataset on which I work: https://www.dropbox.com/s/z5y1q5i29xvki7v/smallsample.sas7bdat?dl=0

Are there any other procedures in SAS to perform random-intercept log linear model?

[ballardw]

Provide data in the form of data step code pasted  into a text box. That way the data is actually available.

Instructions here: https://communities.sas.com/t5/SAS-Communities-Library/How-to-create-a-data-step-version-of-your-dat... will show how to turn an existing SAS data set into data step code that can be pasted into a forum code box using the </> icon or attached as text to show exactly what you have and that we can test code against.

You dropbox link says file deleted.

Show your log, copy the text from the log for the procedure including the code and all notes, warnings or other messages and paste into a text box on the forum.

[Demographer]

I tried the macro to genereate the dataset but got this error message:

1008 *data set you want to create demo data for;
1009 %let dataSetName = test.sample;
1010 *number of observations you want to keep;
1011 %let obsKeep = 100;
1012
1013
1014 ******************************************************
1015 DO NOT CHANGE ANYTHING BELOW THIS LINE
1016 ******************************************************;
1017
1018 %let source_path =
1018! https://gist.githubusercontent.com/statgeek/bcc55940dd825a13b9c8ca40a904cba9/raw/865d2cf18f5150b
1018! 8e887218dde0fc3951d0ff15b/data2datastep.sas;
1019
1020 filename reprex url "&source_path";
1021 %include reprex;
ERROR: Windows SSL error -2146893054 (0x80090302) occurred at line 2320, the error message is "The
function requested is not supported "
ERROR: Windows SSL error -2146893054 (0x80090302) occurred at line 2320, the error message is "The
function requested is not supported ".
ERROR: Cannot open %INCLUDE file REPREX.
1022 filename reprex;
NOTE: Fileref REPREX has been deassigned.
1023
1024 option linesize=max;
1025 %data2datastep(dsn=&dataSetName, obs=&obsKeep);
-
180
WARNING: Apparent invocation of macro DATA2DATASTEP not resolved.

ERROR 180-322: Statement is not valid or it is used out of proper order.

Here is the revised dropbox link:  https://www.dropbox.com/s/hwhuegfzur2smjg/smallsample.sas7bdat?dl=0

[jiltao]

What happens if you try a different estimation method, such as method=quad or method=laplace?

Thanks,

Jill

[Demographer]

In both cases, it says "Model is too large to be fir by PROC GLIMMIX in a reasonable amount of time on this system. Consider changing your model".

From my point of view, my model is already quite simple.

[Demographer]

Update: I managed to have a model that converges by adding TECH=NRRIDG in NLOPTIONS.

proc glimmix data=test.sample INITGLM /*plots=residualpanel*/;
class sample edugr SEXR  ;
model norm_sal= nbedu|edugr agegr SEXR / solution dist=gamma link=log;
random intercept / subject=sample;
weight perwt;
where 0<norm_sal<15;
NLOPTIONS TECH=NRRIDG MAXITER=100 ;
output out=test.rand_intall_pre resid=res pred=pred student=stand;
run;

However when I add a statement to create an output dataset with predicted values and residuals such as the code above, it does not converge anymore and I'm very confused since I thought this was completely unrelated. 

I also tried to add "plots=residualpanel" but I believe I have too many observations. Is it possible to reduce the number of points on graphs?

[JoséQ]

Hi,

It does converge for me.

Are you using the same data set you made available through the latest dropbox link ?

Can you run the bellow code and send the resulting log:

%put Release: &sysvlong    System: &sysscp &sysscpl;

[Demographer]

Sorry I'm not sure how to use this code properly. I run it both before and after proc glimmix but the log still does not say much except that it does not converge. Also: it did also happen to me sometimes that the exact same model with the same data converges. It is just not consistent.

1 %put Release: &sysvlong System: &sysscp &sysscpl;
Release: 9.04.01M2P072314 System: WIN X64_8PRO
2
3 libname test2 "C:\Users\Downloads";
NOTE: Libref TEST2 was successfully assigned as follows:
Engine: V9
Physical Name: C:\Users\marois\Downloads
4
NOTE: Writing HTML Body file: sashtml.htm
5 proc glimmix data=test2.smallsample INITGLM;
6 class edugr SEXR sample ;
7 model norm_sal= nbedu|edugr agegr SEXR / solution dist=gamma link=log;
8 random intercept / subject=sample;
9 weight perwt;
10 NLOPTIONS MAXITER=200 ;
11 run;

NOTE: Some observations are not used in the analysis because of: missing response values (n=83136),
zero or negative response (n=258).
NOTE: Did not converge.
NOTE: PROCEDURE GLIMMIX used (Total process time):
real time 1:45.00
cpu time 1:44.29

12 %put Release: &sysvlong System: &sysscp &sysscpl;
Release: 9.04.01M2P072314 System: WIN X64_8PRO

[JoséQ · Accepted solution]

Hi,

The code I sent was just to know the SAS version you have installed.

The inconsistency you are observing, which I have replicated, is related with the older version of the software you have installed, SAS 9.4M2 dates back to 2014.  

Upgrading to a newer version, 9.4M8 is the latest, will solve your problem.

[Demographer]

Thanks, I just checked in my version is indeed SAS 9.4 TS Level 1M2, so I guess this is not the latest one. I'll check with the admin to upgrade it, hoping it will fix this issue.