topic Re: Overwriting inside a macro loop leads to Out of memory for symbols ERROR in SAS/IML Software and Matrix Computations

Overwriting inside a macro loop leads to Out of memory for symbols ERROR

Rabelais — Thu, 08 Jan 2026 09:47:50 GMT

Hi, I'm using SAS Enterprise Guide version 8.3 Update 3 (8.3.3.181) (64-bit).

I noticed a strange bug when overwriting a variable thousands of times inside a macro loop (normal loops don't have this problem). This is just a toy example reproducing the bug, but let's say we need to overwrite a variable ("b" in the following codes) many many times with itself and we have to use a macro loop. Firstly let's see the minimal memory usage (0 iterations/overwrites)

%macro test;
proc iml;
a = j(1000,1,1);
b = a;
quit;
%mend;
%test;

and this is the log

      memory              544.25k
      OS Memory           21088.00k

By gradually increasing the overwrites we see an increase in the memory usage. With 1 million overwrites

%macro test;
proc iml;
a = j(1000,1,1);
%do i=1 %to 1000000;
	b = a;
%end;
quit;
%mend;
%test;

the memory usage is increased by more than 200 times

      memory              125586.25k
      OS Memory           163680.00k

With this slightly different code and by increasing the overwrites even more

%macro loop;
proc iml;
a = j(10000,1,42);
%do i=1 %to 3000000;
	%do j=1 %to 100;
		simulated_loop_&j = a*3;
	%end;
%end;
quit;
%mend;
%loop;

after 50 minutes SAS raised ERROR: Out of memory for symbols. Cannot proceed.

This is the (truncated) log

1                                         The SAS System              11:26 Monday, January 5, 2026

1          ;*';*";*/;quit;run;
2          OPTIONS LS=99 PS=32767;
3          OPTIONS PAGENO=MIN;
4          %LET _CLIENTTASKLABEL='Program 1';
5          %LET _CLIENTPROCESSFLOWNAME='Standalone Not In Project';
6          %LET _CLIENTPROJECTPATH='';
7          %LET _CLIENTPROJECTPATHHOST='';
8          %LET _CLIENTPROJECTNAME='';
9          %LET _SASPROGRAMFILE='';
10         %LET _SASPROGRAMFILEHOST='';
11         
12         ODS _ALL_ CLOSE;
13         OPTIONS DEV=SVG;
14         GOPTIONS XPIXELS=0 YPIXELS=0;
15         %macro HTML5AccessibleGraphSupported;
16             %if %_SAS_VERCOMP_FV(9,4,4, 0,0,0) >= 0 %then ACCESSIBLE_GRAPH;
17         %mend;
18         FILENAME EGHTML TEMP;
19         ODS HTML5(ID=EGHTML) FILE=EGHTML
20             OPTIONS(BITMAP_MODE='INLINE')
21             %HTML5AccessibleGraphSupported
22             ENCODING='utf-8'
23             STYLE=HTMLEncore
24             GPATH=&sasworklocation
25         ;
NOTE: Writing HTML5(EGHTML) Body file: EGHTML
26         
27         
28         %macro loop;
29         
30         proc iml;
31         
32         a = j(10000,1,42);
33         
34         %do i=1 %to 3000000;
35         	%do j=1 %to 100;
36         		simulated_loop_&j = a*3;
37         	%end;
38         %end;
39         
40         quit;
41         
42         %mend;
43         
44         %loop;
NOTE: IML Ready
NOTE: The internal source spool file has been truncated. Error logging with line and column 
      information might be incomplete until next step boundary.
NOTE: The internal source spool file has been truncated. Error logging with line and column 
      information might be incomplete until next step boundary.
NOTE: The internal source spool file has been truncated. Error logging with line and column 
      information might be incomplete until next step boundary.
NOTE: The internal source spool file has been truncated. Error logging with line and column 
      information might be incomplete until next step boundary.
NOTE: The internal source spool file has been truncated. Error logging with line and column 
      information might be incomplete until next step boundary.
NOTE: The internal source spool file has been truncated. Error logging with line and column 
      information might be incomplete until next step boundary.
NOTE: The internal source spool file has been truncated. Error logging with line and column 
      information might be incomplete until next step boundary.
NOTE: The internal source spool file has been truncated. Error logging with line and column 
      information might be incomplete until next step boundary.
NOTE: The internal source spool file has been truncated. Error logging with line and column 
      information might be incomplete until next step boundary.
NOTE: The internal source spool file has been truncated. Error logging with line and column 
      information might be incomplete until next step boundary.
NOTE: The internal source spool file has been truncated. Error logging with line and column 
      information might be incomplete until next step boundary.
NOTE: The internal source spool file has been truncated. Error logging with line and column 
      information might be incomplete until next step boundary.
NOTE: The internal source spool file has been truncated. Error logging with line and column 
      information might be incomplete until next step boundary.
NOTE: The internal source spool file has been truncated. Error logging with line and column 
      information might be incomplete until next step boundary.
NOTE: The internal source spool file has been truncated. Error logging with line and column 
      information might be incomplete until next step boundary.
NOTE: The internal source spool file has been truncated. Error logging with line and column 
      information might be incomplete until next step boundary.
NOTE: The internal source spool file has been truncated. Error logging with line and column 
      information might be incomplete until next step boundary.
NOTE: The internal source spool file has been truncated. Error logging with line and column 
      information might be incomplete until next step boundary.
NOTE: The internal source spool file has been truncated. Error logging with line and column 
      information might be incomplete until next step boundary.
NOTE: The internal source spool file has been truncated. Error logging with line and column 
      information might be incomplete until next step boundary.
NOTE: The internal source spool file has been truncated. Error logging with line and column 
      information might be incomplete until next step boundary.
NOTE: The internal source spool file has been truncated. Error logging with line and column 
      information might be incomplete until next step boundary.
NOTE: The internal source spool file has been truncated. Error logging with line and column 
      information might be incomplete until next step boundary.
NOTE: The internal source spool file has been truncated. Error logging with line and column 
      information might be incomplete until next step boundary.
NOTE: PROCEDURE IML used (Total process time):
      real time           50:16.32
      user cpu time       48:47.31
      system cpu time     1:26.51
      memory              58767352.00k
      OS Memory           67108696.00k
      Timestamp           01/05/2026 02:20:54 PM
      Step Count                        65  Switch Count  5
      
ERROR: Out of memory for symbols. Cannot proceed.
NOTE: Exiting IML.

180: LINE and COLUMN cannot be determined.
NOTE: NOSPOOL is on. Rerunning with OPTION SPOOL might allow recovery of the LINE and COLUMN where 
      the error has occurred.
ERROR 180-322: Statement is not valid or it is used out of proper order.
180: LINE and COLUMN cannot be determined.
NOTE: NOSPOOL is on. Rerunning with OPTION SPOOL might allow recovery of the LINE and COLUMN where 
      the error has occurred.
ERROR 180-322: Statement is not valid or it is used out of proper order.
180: LINE and COLUMN cannot be determined.
NOTE: NOSPOOL is on. Rerunning with OPTION SPOOL might allow recovery of the LINE and COLUMN where 
      the error has occurred.
ERROR 180-322: Statement is not valid or it is used out of proper order.

truncated (these lines are repeated million of times)

As you can see the memory increased to 58 GB and then SAS raised the error. The number of symbols is 101 so I don't understand why it raises a "Out of memory for symbols" rather than a "Unable to allocate sufficient memory", anyway clearly there is a problem here. Moreover, the last 4 lines of the log are repeated millions of times so this creates a huge logs (I got a 7 GB log with a similar code).

Interestingly, there are no memory problems by using normal loops rather than macro loops. The following code

%macro test;
proc iml;
a = j(1000,1,1);
do i=1 to 1000000;
	b = a;
end;
quit;
%mend;
%test;

uses

      memory              561.87k
      OS Memory           21088.00k

which is just slightly greater than the minimal memory usage (544.25k as shown in the first log above).

Is the memory increase an expected behavior or a bug?

Re: Overwriting inside a macro loop leads to Out of memory for symbols ERROR

Rick_SAS — Thu, 08 Jan 2026 11:22:28 GMT

What version of SAS are you using? Run
%put &=SYSVLONG;
and post the result from the Log.

Re: Overwriting inside a macro loop leads to Out of memory for symbols ERROR

Rabelais — Thu, 08 Jan 2026 13:39:20 GMT

SYSVLONG=9.04.01M7P080520

Re: Overwriting inside a macro loop leads to Out of memory for symbols ERROR

Rick_SAS — Thu, 08 Jan 2026 15:01:02 GMT

I believe what you are seeing is that parsing and compiling a SAS program requires memory.

Recall that a macro call is a PREPROCESSOR. It generates test, which are usually SAS statements. In the '%macro loop' example, the macro generates a program that contains 300 MILLION statements. It doesn't matter what the statements are, what matters is that there are 300 million statements.

This huge program is then sent to PROC IML for parsing and execution. The parsing is performed in C. Parsing the program results in a (huge!) C structure that represents the program. That structure is then sent to the procedure for execution. If the amount of memory exceeds the MEMSIZE option in your SAS session, SAS will report an out-of-memory error, which is what is happening in your example.

Your example isn't limited to PROC IML. You can see the same behavior in the DATA step if you use macro to generate a program that contains hundreds of millions of lines. I ran the following macro code to generate huge programs in both the DATA step and PROC IML.

options fullstimer;
options nosource;

%macro IMLtest(numStmts);
%put Test IML: numStmts=%sysfunc(putn(&numStmts,comma9.));
proc iml;
a = 1;
%do i=1 %to &numStmts;
	b = a;
%end;
quit;
%mend;
%put ----- START IML TESTS -----;
%IMLtest(10000);
%IMLtest(100000);
%IMLtest(500000);
%IMLtest(1000000);

%macro DStest(numStmts);
%put Test DATA Step: numStmts=%sysfunc(putn(&numStmts,comma9.));
data _NULL_;
a = 1;
%do i=1 %to &numStmts;
	b = a;
%end;
run;
%mend;
%put ----- START DATA STEP TESTS -----;
%DStest(10000);
%DStest(100000);
%DStest(500000);
%DStest(1000000);

To make the programs comparable, I use 'a=1' in IML and in the DATA step. If 'a' is a vector in IML, then IML will use a little more memory, but it is really the length of the program (known at compile time) that is causing this issue, not the run-time memory.

On my PC version of SAS, the memory usage for each PROC is as follows:

data VizMemory;
length Proc $10;
input Proc numStmts Memory;
label numStmts="Number of Statements" Memory="Memory (k)";
format numStmts Memory comma10.;
datalines;
DATA_STEP 10000 3907
DATA_STEP 100000 37787
DATA_STEP 500000 180044
DATA_STEP 1000000 359641
IML 1000 254
IML 10000 1428
IML 100000 12698
IML 500000 62705
IML 1000000 125271
;

title "Memory Usage to Parse and Compile SAS Programs";
proc sgplot data=VizMemory;
  series x=numStmts y=Memory / group=Proc markers;
  xaxis grid;
  yaxis grid;
run;

The memory usage scales linearly with the number of SAS statements in the programs. The DATA step actually uses more memory than PROC IML to parse and execute a similar program.

So, in conclusion, you are seeing the result of generating a program that is hundreds of millions of lines long. Those text statements are parsed, which converts the text to a C structure, which requires considerable memory. (Think about a linked list that has 300 million items, and each item has information about the statement and the symbols.) My advice is to use the loops and conditional logic in IML to write your program, rather than using the macro preprocessor to generate repeated statements.

Re: Overwriting inside a macro loop leads to Out of memory for symbols ERROR

Tom — Thu, 08 Jan 2026 15:03:37 GMT

By using macro code to generate more STATEMENTS you are changing the size of the program.

What happens if you instead generate the statements to a text file and ask PROC IML to %INCLUDE to execute those 300 million statements?

Re: Overwriting inside a macro loop leads to Out of memory for symbols ERROR

Rabelais — Thu, 08 Jan 2026 16:39:14 GMT

Many thanks for the detailed analysis! So basically are you saying that these lines of code

%do i=1 %to 1000000; b = a; %end;

are turned into 1000000 lines of text where each line is equal to "b = a;" ?

Re: Overwriting inside a macro loop leads to Out of memory for symbols ERROR

Tom — Thu, 08 Jan 2026 16:43:11 GMT

@Rabelais wrote:

Many thanks for the detailed analysis! So basically are you saying that these lines of code
%do i=1 %to 1000000; b = a; %end;
are turned into 1000000 lines of text where each line is equal to "b = a;" ?

Yes. That is how a macro PRE-processor works. And SAS's macro is no different. It intercepts the program and sends the results on to SAS to actually execute.

Re: Overwriting inside a macro loop leads to Out of memory for symbols ERROR

Rick_SAS — Thu, 08 Jan 2026 16:46:17 GMT

Of course. The SAS macro facility generates text, usually SAS statements.

Re: Overwriting inside a macro loop leads to Out of memory for symbols ERROR

ballardw — Fri, 09 Jan 2026 06:02:27 GMT

@Rabelais wrote:

Many thanks for the detailed analysis! So basically are you saying that these lines of code
%do i=1 %to 1000000; b = a; %end;
are turned into 1000000 lines of text where each line is equal to "b = a;" ?

The SAS option MPRINT, assuming no syntax problems will show the generated code in the LOG.

Example:

%macro loop;
%do i=1 %to 25;
   %put Loop is &i. ;
%end;
%mend;

options mprint;
%loop

The log will show the result of each %put statement, only 25 in this case. I used that for an example to avoid the errors that would occur with assignments when no data step is present.

Turn off the option with :

options nomprint;

after the code.

The options SYMBOLGEN will show the results of creating and combining macro variable values, MLOGIC will show the result of macro logic comparisons such as %if statements.