PROC GLIMMIX and insufficient memory

[rchung]

Hello.
I am running SAS 9.4 / SAS-EG 8.3 under Windows 10 Enterprise 64-bit OS. I have 32GB of installed RAM. When no application is running, the task manager shows that 17GB is available. 

I want to run 3-level random-intercept models with a count response variable (i.e., number of cases diagnosed with a certain disease) that takes the Poisson distribution. 

• The level-3 variable is county: There are ~60 counties.
• The level-2 variable is census tract: There are ~5,000 census tracts.
• The sample size is ~200,000.

I have tried a null model:

• proc glimmix data=m100 method=laplace;
  class tract county ;
  model n_case=  /dist=poisson link=log offset=logpop solution;
  random intercept / subject=county type=chol;
  random intercept / subject=tract(county) type=chol;
  run;

Unfortunately, PROC GLIMMIX stops running due to ERROR: The SAS System stopped processing this step because of insufficient memory.

I have tried these configurations in the sasv9.cfg file:

• -MEMSIZE MAX
  -FULLSTIMER
  
  
• -MEMSIZE 24G
  -FULLSTIMER
  
  
• -MEMSIZE 40G                     --> so that SAS could use HDD space as virtual memory?
  -REALMEMSIZE 24G
  -FULLSTIMER

Unfortunately, PROC GLIMMIX kept stopping in 10 seconds and showing the same error. When I tried METHOD=QUAD(FASTQUAD), I ran into the same error.

Then, I started running the same model on smaller data. The table below shows the data size and a couple FULLSTIMER stats.

data	n	tract	county	real time	OS memory	error
m40	69584	1848	25	32 min	3153220.00k
m64	116715	3104	40	80 min	5326244.00k
m80	154798	4119	50	135 min	7330232.00k
m88	170314	4528	55	161 min	8499144.00k
m96	182635	4854	60	200 min	9780144.00k
m98	183091	4866	61	stops at 41 min	10050376.00k	write access violation
m100	183243	4870	62	stops at 10 sec		insufficient memory

It was interesting to find this ERROR: Write Access Violation GLIMMIX (2) when I ran the model on a nearly full data.

• I wonder that only 10GB of memory was available for SAS, despite the configurations.
• I also wonder that HDD space was not available for SAS to use as virtual memory because I do not have the admin privileges on this computer.

Now, I wonder how much of additional RAM will do the job. Any suggestions will be greatly appreciated.

Thank you!
Rakkoo

[ChrisNZ]

Strange that SAS would fail at 10 GB when you allocate much more.

See the value of undocumented option XMRLMEM to know the amount of memory available to SAS at any point.

[ChrisNZ]

How much memory do you see available in Windows?

[rchung]

For now, I am not running any other apps but SAS and Chrome. Task manager reports memory usage as below:

In use: 12.0 GB

Available: 19.6 GB

[rchung]

I ran this:

• DATA _NULL_; mem = (GETOPTION('XMRLMEM')); PUT '**mem=' mem; RUN;

Available memory when other apps (Outlook, Chrome, Excel, Word, PPT, etc., as well as SAS and Chrome) are running:

• **mem=19360390144

Available memory when no other apps (but SAS and Chrome) are running:

• **mem=19360600064

[ChrisNZ]

So you have 20 GB available. Contact Tech suport. There's no reason SAS should stop at 10 GB. There are other memory settings like SORTSIZE and SUMSIZE, but they don't affect this procedure. Tech support might know more. Now you have good information to give them.