Hello,
I have received the following error message:
ERROR: The SAS System stopped processing this step because of insufficient memory.
NOTE: PROCEDURE SURVEYREG used (Total process time):
real time 21.09 seconds
cpu time 21.15 seconds
To put this into context, I am trying to run proc survey reg clustering errors along two dimensions with ~5 million observations. The macro I am using for this is posted below. I am using SAS 9.4 on Windows 10 Pro. My computer has 32 GB of ram and I believe I have set my memory options appropriately...I right clicked my SAS 9.4 icon, chose properties, and ammended the "target" line to say
"C:\Program Files\SASHome\SASFoundation\9.4\sas.exe" -CONFIG "C:\Program Files\SASHome\SASFoundation\9.4\nls\en\sasv9.cfg" -MEMSIZE 30G
Essentially just adding the part in bold.
The macro I used is listed below and I am trying to include 16 independent variables. I find it hard to imagine that 30GB of memory is not sufficient to complete this task.
Thanks!
%MACRO clus2OLS(yvar, xvars, cluster1, cluster2, dset);
/* do interesection cluster*/
proc surveyreg data=&dset; cluster &cluster1 &cluster2; model &yvar= &xvars / covb; ods output CovB = CovI; quit;
/* Do first cluster */
proc surveyreg data=&dset; cluster &cluster1; model &yvar= &xvars / covb; ods output CovB = Cov1; quit;
/* Do second cluster */
proc surveyreg data=&dset; cluster &cluster2; model &yvar= &xvars / covb; ods output CovB = Cov2 ParameterEstimates = params; quit;
/* Now get the covariances numbers created above. Calc coefs, SEs, t-stats, p-vals using COV = COV1 + COV2 - COVI*/
proc iml; reset noprint; use params;
read all var{Parameter} into varnames;
read all var _all_ into b;
use Cov1; read all var _num_ into x1;
use Cov2; read all var _num_ into x2;
use CovI; read all var _num_ into x3;
cov = x1 + x2 - x3; /* Calculate covariance matrix */
dfe = b[1,3]; stdb = sqrt(vecdiag(cov)); beta = b[,1]; t = beta/stdb; prob = 1-probf(t#t,1,dfe); /* Calc stats */
print,"Parameter estimates",,varnames beta[format=8.4] stdb[format=8.4] t[format=8.4] prob[format=8.4];
conc = beta || stdb || t || prob;
cname = {"estimates" "stderror" "tstat" "pvalue"};
create clus2dstats from conc [ colname=cname ];
append from conc;
conc = varnames;
cname = {"varnames"};
create names from conc [ colname=cname ];
append from conc;
quit;
data clus2dstats; merge names clus2dstats; run;
%MEND clus2OLS;
Try running with system option: -memsize = 0;
That will enable use memory and virtual memory as much as needed.
How do I set a system option?
@starky987 wrote:
How do I set a system option?
Find the -memsize option in the config file and change the value to zero
or in your sas execution:
"C:\Program Files\SASHome\SASFoundation\9.4\sas.exe"
-CONFIG "C:\Program Files\SASHome\SASFoundation\9.4\nls\en\sasv9.cfg" -MEMSIZE 0
I went ahead and made that change with no luck. I've posted the fullstimer results and I think (not sure though) it is an issue related to the OS memory from that log?
I'm not entirely sure, I just know its not possible that this file takes up more than the 32G of ram available.
According to the fullstimer log, you had less that 1GB memory in use. Could it be that there is a system setting (like the UNIX ulimit) in place that limits the resources available for your SAS process?
Run your program with the SAS option FULLSTIMER and the SAS log will report how much memory is being used in each step.
I received the following information from fulltimer
NOTE: PROCEDURE SURVEYREG used (Total process time):
real time 21.64 seconds
user cpu time 19.18 seconds
system cpu time 2.48 seconds
memory 572712.68k
OS Memory 652840.00k
Timestamp 02/16/2017 03:26:20 PM
Step Count 73 Switch Count 0
It doesn't help you solve this with PROC SURVEYREG, but I've had a similar issue in the past and was not able to resolve through the suggested methods. I think they work in the right environment. I'm using a server-delivered SAS and therefore do not have complete control over how much memory (or at least, that's my working theory).
I've ended up using a different approach: Create a random number as a variable in that data set, and then do a selection off that. So for instance, create a random number from 0 to 1, then only output when the value is less than 0.05 (5% sample).
I have to admit that my Windows admin skills are virtually nonexistent (compared to my AIX admin skills). Even googling did not provide me with a hint.
Are you ready for the spotlight? We're accepting content ideas for SAS Innovate 2025 to be held May 6-9 in Orlando, FL. The call is open until September 25. Read more here about why you should contribute and what is in it for you!
Learn how use the CAT functions in SAS to join values from multiple variables into a single value.
Find more tutorials on the SAS Users YouTube channel.