Insufficient memory

[Naviava1973]

Hi,

I am trying to run a propensity score match on a large dataset (40K IDs) and getting the following error:

ERROR: The SAS System stopped processing this step because of insufficient memory.

WARNING: The data set WORK.OUT_Out1 may be incomplete.  When this step was stopped there were 0 observations and 153 variables.

WARNING: Data set WORK.OUT_Out1 was not replaced because this step was stopped.

Is there a way I can increase my SAS Enterprise Guide 7.15 memory from 2GB to 6GB or 8GB?

I have tried including an options statement prior to running my code without success. I still get the same error.

My code is below:

options memsize=0;

proc psmatch data=dataset region=cs;    

     class HIV TB;

     psmodel HIV(treated="Positive")=TB;

     match method=optimal(k=1) stat=lps exact=TB caliper=0.25;

     output out(obs=match)=Out1 matchid=_matchid;

run;

Could someone advise on the best option to increase the memory size.

Regards

Otwombe

[SASKiwi]

Does Enterprise Guide point to SAS on your PC or to a remote SAS server? If it is the latter then talk to your SAS administrator as they have to change a SAS server start-up setting to increase memory.

[Naviava1973]

The enterprise Guide points to my laptop and not the server. 

[SASKiwi]

In that case the MEMSIZE option needs to be set during startup by adding a line to your SAS config file. For example:

- memsize 8G

[Naviava1973]

Hi SASKiwi,

I have made the adjustment in the sasv9.cfg file on the path C:\Program Files\SASHome\SASFoundation\9.4\nls\en as advised but without success.

The memory size is now 8G i.e MEMSIZE 8G.

I have run the code again and got the same error:

ERROR: The SAS System stopped processing this step because of insufficient memory.
WARNING: The data set WORK.OUT1 may be incomplete. When this step was stopped there were 0 observations and 153 variables.

[SASKiwi]

You could try running your program with the FULLSTIMER SAS option to get more details of the memory used.

It is more than likely that your program is simply too complex for the amount of memory available. I would try to limit the data volume first to see if that helps. Failing that try simplifying the model. I'm not familiar with the PSMATCH procedure so I'll leave it for someone more qualified to answer.

[MichaelL_SAS · Accepted solution]

Instead of using the METHOD=OPTIMAL(k=1) matching method you could try using METHOD=GREEDY(k=1).

The optimal matching methods produce a matched data set that minimizes the total absolute difference in the matching distance across all feasible matches. The optimal matching methods are therefore the most computationally and resource intensive methods supported by PROC PSMATCH and are likely to return an out of memory error for larger data sets due to the number of distances that must be computed and stored in memory.

Also looking at your example code, because the propensity score model includes only the categorical variable TB and the EXACT=TB option is specified, all matched subjects will have the same propensity score value regardless of which matching method is used.