BookmarkSubscribeRSS Feed
🔒 This topic is solved and locked. Need further help from the community? Please sign in and ask a new question.
Naviava1973
Calcite | Level 5

Hi,

I am trying to run a propensity score match on a large dataset (40K IDs) and getting the following error:

 

ERROR: The SAS System stopped processing this step because of insufficient memory.

WARNING: The data set WORK.OUT_Out1 may be incomplete.  When this step was stopped there were 0 observations and 153 variables.

WARNING: Data set WORK.OUT_Out1 was not replaced because this step was stopped.

 

Is there a way I can increase my SAS Enterprise Guide 7.15 memory from 2GB to 6GB or 8GB?

 

I have tried including an options statement prior to running my code without success. I still get the same error.

My code is below:

 

options memsize=0;

proc psmatch data=dataset region=cs;    

     class HIV TB;

     psmodel HIV(treated="Positive")=TB;

     match method=optimal(k=1) stat=lps exact=TB caliper=0.25;

     output out(obs=match)=Out1 matchid=_matchid;

run;

 

Could someone advise on the best option to increase the memory size.

 

Regards

Otwombe

1 ACCEPTED SOLUTION

Accepted Solutions
MichaelL_SAS
SAS Employee

Instead of using the METHOD=OPTIMAL(k=1) matching method you could try using METHOD=GREEDY(k=1).

 

The optimal matching methods produce a matched data set that minimizes the total absolute difference in the matching distance across all feasible matches. The optimal matching methods are therefore the most computationally and resource intensive methods supported by PROC PSMATCH and are likely to return an out of memory error for larger data sets due to the number of distances that must be computed and stored in memory.

 

Also looking at your example code, because the propensity score model includes only the categorical variable TB and the EXACT=TB option is specified, all matched subjects will have the same propensity score value regardless of which matching method is used.

View solution in original post

6 REPLIES 6
SASKiwi
PROC Star

Does Enterprise Guide point to SAS on your PC or to a remote SAS server? If it is the latter then talk to your SAS administrator as they have to change a SAS server start-up setting to increase memory.

Naviava1973
Calcite | Level 5

The enterprise Guide points to my laptop and not the server. 

SASKiwi
PROC Star

In that case the MEMSIZE option needs to be set during startup by adding a line to your SAS config file. For example:

- memsize 8G

Naviava1973
Calcite | Level 5

Hi SASKiwi,

I have made the adjustment in the sasv9.cfg file on the path C:\Program Files\SASHome\SASFoundation\9.4\nls\en as advised but without success.

The memory size is now 8G i.e MEMSIZE 8G.

 

I have run the code again and got the same error:


ERROR: The SAS System stopped processing this step because of insufficient memory.
WARNING: The data set WORK.OUT1 may be incomplete. When this step was stopped there were 0 observations and 153 variables.

SASKiwi
PROC Star

You could try running your program with the FULLSTIMER SAS option to get more details of the memory used.

 

It is more than likely that your program is simply too complex for the amount of memory available. I would try to limit the data volume first to see if that helps. Failing that try simplifying the model. I'm not familiar with the PSMATCH procedure so I'll leave it for someone more qualified to answer.

MichaelL_SAS
SAS Employee

Instead of using the METHOD=OPTIMAL(k=1) matching method you could try using METHOD=GREEDY(k=1).

 

The optimal matching methods produce a matched data set that minimizes the total absolute difference in the matching distance across all feasible matches. The optimal matching methods are therefore the most computationally and resource intensive methods supported by PROC PSMATCH and are likely to return an out of memory error for larger data sets due to the number of distances that must be computed and stored in memory.

 

Also looking at your example code, because the propensity score model includes only the categorical variable TB and the EXACT=TB option is specified, all matched subjects will have the same propensity score value regardless of which matching method is used.

sas-innovate-2024.png

Don't miss out on SAS Innovate - Register now for the FREE Livestream!

Can't make it to Vegas? No problem! Watch our general sessions LIVE or on-demand starting April 17th. Hear from SAS execs, best-selling author Adam Grant, Hot Ones host Sean Evans, top tech journalist Kara Swisher, AI expert Cassie Kozyrkov, and the mind-blowing dance crew iLuminate! Plus, get access to over 20 breakout sessions.

 

Register now!

SAS Enterprise Guide vs. SAS Studio

What’s the difference between SAS Enterprise Guide and SAS Studio? How are they similar? Just ask SAS’ Danny Modlin.

Find more tutorials on the SAS Users YouTube channel.

Click image to register for webinarClick image to register for webinar

Classroom Training Available!

Select SAS Training centers are offering in-person courses. View upcoming courses for:

View all other training opportunities.

Discussion stats
  • 6 replies
  • 9310 views
  • 1 like
  • 3 in conversation