topic Re: Insufficient memory error, but the obvious solutions aren't working in Statistical Procedures

Insufficient memory error, but the obvious solutions aren't working

hsmyth — Tue, 18 Aug 2020 03:21:37 GMT

Using SAS Software 9.4 (TS1M5)

X64_10HOME platform.

SAS/STAT 14.3

Program file and data are attached

Yesterday, I ran a program using the default 2GB Memsize option. Today, when I return to the program I am getting the following error:

ERROR: Insufficient memory to complete the CAUSALMED procedure. ERROR: The SAS System stopped processing this step because of insufficient memory.

Here is an example of the program that worked ysterday, but not today. I can narrow the problem down to the bootstrap command. The program works if I leave that part out.

options fullstimer;
ods results = OFF;
ods listing close;
ods html close;
ods output EffectDecomp=EffectDecomp EffectSummary=EffectSummary MediatorEstimates=MediatorEstimates OutcomeEstimates=OutcomeEstimates PercentDecomp=PercentDecomp;
PROC CAUSALMED DATA=Simdata.f163 pALL;
BY rep;
CLASS X/DESCENDING;
MODEL Y = X|M;
MEDIATOR M = X;
BOOTSTRAP BOOTCI(NORMAL) NBOOT=1000 SEED=100;
quit;
ods output close;
ods listing;
ods html;
run;

I have spent all day reading through forums and documentation, I have changed MEMSIZE option, and I have confirmed that I have over 7GB memory available. (Remember, this worked yesterday when the MEMSIZE was at the default. The FULLSTIMER option suggests that only 2200 k is being used.

Here is the full log

4 options fullstimer;
5 ods results = OFF;
6 ods listing close;
7 ods html close;
8 ods output EffectDecomp=EffectDecomp EffectSummary=EffectSummary
8 ! MediatorEstimates=MediatorEstimates OutcomeEstimates=OutcomeEstimates PercentDecomp=PercentDecomp
8 ! ;
9 PROC CAUSALMED DATA=Simdata.f163 pALL;
10 BY rep;
11 CLASS X/DESCENDING;
12 MODEL Y = X|M;
13 MEDIATOR M = X;
14 BOOTSTRAP BOOTCI(NORMAL) NBOOT=1000 SEED=100;
15 quit;

NOTE: Resampling for bootstrap replicates 1 to 100.
ERROR: Insufficient memory to complete the CAUSALMED procedure.
ERROR: The SAS System stopped processing this step because of insufficient memory.
NOTE: PROCEDURE CAUSALMED used (Total process time):
real time 0.05 seconds
user cpu time 0.03 seconds
system cpu time 0.06 seconds
memory 2201.46k
OS Memory 18932.00k
Timestamp 08/17/2020 08:02:17 PM
Step Count 1 Switch Count 0

16 ods output close;
17 ods listing;
18 ods html;
NOTE: Writing HTML Body file: sashtml.htm
19 run;

I have cleared the work folder, checked my computer memory, I have updated the configuration file, deleted temporary files from C:\User..., I have tried all the things that seem obvious to do given the error message. Nothing worked.

I remoted into a different work computer (which would have the default MEMSIZE) and ran the program with the bootstrap with no problem. But my home desktop and laptop give me this error.

Does anyone have any ideas about what is going on? It seems like something is clogged up in the machine, but I don't know where else to look.

Thanks.

Re: Insufficient memory error, but the obvious solutions aren't working

FreelanceReinh — Tue, 18 Aug 2020 09:26:05 GMT

Hello @hsmyth and welcome to the SAS Support Communities!

Thanks for providing usable code and input data for testing. I got the same error message on my workstation with 64 GB RAM (part of which is used for a RAM disk, though) and -MEMSIZE MAX. I also modified the REALMEMSIZE system option to no avail. The error occurs even with NBOOT=50 applied to Simdata.f163(obs=4). I'm not familiar with PROC CAUSALMED, but I can't imagine how 50 bootstrap samples from 4 observations (of length 144 bytes) could ever occupy anything near 2 gigabytes.

To try another example I added a similar BOOTSTRAP statement with NBOOT=50 to (simplified versions of) Example 35.3 from the documentation. Then the error messages ranged from "Insufficient memory" over "Floating Point Overflow" to "Write Access Violation CAUSALMED" and "An exception has been encountered. Please contact technical support ...".

The example in Usage Note 59081: Mediation analysis (see Re: SAS housing data for the data), however, worked on my computer -- until I changed the default BOOTCI(BC) in the BOOTSTRAP statement to BOOTCI(NORMAL). This change caused "An exception has been encountered. Please contact technical support ..."

So, this is what I would do next: contact SAS technical support. There is likely a bug in this new procedure (which was introduced with SAS/STAT 14.3).

Re: Insufficient memory error, but the obvious solutions aren't working

hsmyth — Wed, 19 Aug 2020 03:26:06 GMT

Thank you so much for taking the time to look at this. I will check in with the support team and see what might be going on.