Insufficient memory when matching table

[Tom]

I suspect that what SAS is showing for the memory settings is what SAS will limit itself to using.

But the error is coming because the operating system cannot actually allocate the amount of memory that is needed.  Perhaps other users are using it.  Or other programs if running on a PC.

[Patrick]

@Tom It's the memsize option set to two GB that limits things. But what astonishes me is that @ballardw says things work for him even though he's got the same setting of 2GB.

[ballardw]

@Patrick 

I am running SAS 9.4.4 on Windows 10. System has 8Gb installed and task manager shows basically 4.7Gb "in use". Running the following data step that goes to 5.3 temporarily then returns to 4.7.

When I assign a value similar to yours I do not get the compiler error and the memory settings are the same as yours

13   data junk;
14      set sashelp.class;
15      array t (41000000) $1 _temporary_ (41000000*'a');
16   run;

NOTE: There were 19 observations read from the data set SASHELP.CLASS.
NOTE: The data set WORK.JUNK has 19 observations and 5 variables.
NOTE: DATA statement used (Total process time):
      real time           0.97 seconds
      cpu time            0.96 seconds


17
18   proc options group=memory;run;

    SAS (r) Proprietary Software Release 9.4  TS1M4


Group=MEMORY
 SORTSIZE=1073741824
                   Specifies the amount of memory that is available to the
                   SORT procedure.
 SUMSIZE=0         Specifies a limit on the amount of memory that is
                   available for data summarization procedures when class
                   variables are active.
 MAXMEMQUERY=0     Specifies the maximum amount of memory that is
                   allocated for procedures.
 MEMBLKSZ=16777216 Specifies the memory block size for Windows
                   memory-based libraries.
 MEMMAXSZ=2147483648
                   Specifies the maximum amount of memory to allocate for
                   using memory-based libraries.
 LOADMEMSIZE=0     Specifies a suggested amount of memory that is needed
                   for executable programs loaded by SAS.
 MEMSIZE=2147483648
                   Specifies the limit on the amount of virtual memory
                   that can be used during a SAS session.
 REALMEMSIZE=0     Specifies the amount of real memory SAS can expect to
                   allocate.

---

@Patrick wrote:

@ballardw What you are sharing is really interesting and also baffles me. I thought that a temporary array will get loaded into memory and though your sample code should require more than 2GB of memory.

I've tested the code you shared under SAS9.4M7 both with a local SAS under Windows and a server instance under RHEL. In both cases more than 2GB of memory is required for not getting an error during compilation time.

Another interesting observation:

With a 2GB memsize limit I can run below code without error.

options fullstimer;
data _null_;
  array t (36000000) $1 _temporary_ ;
run;

But if I populate the array vars then I get again the compiler error with the insufficient memory message.

options fullstimer;
data _null_;
  array t (36000000) $1 _temporary_ (36000000*'a');
run;

When I run above two version on my rhel server environment with a higher memsize limit then the SAS log tells me that the 2nd version consumes more memory (memory 2285623.59k).

Really curious why things appear to work for you. What are your other memory settings? Below are mine on the local Windows environment.

---

[Patrick]

@ballardw What would be interesting to see is what memory usage the SAS log shows if running with option fullstimer.

If the log shows more than the limit set by option memsize then I'd consider that a bug likely fixed in later maintenance releases. ...but if the log shows memory usage below 2GB then it would really be interesting to figure out which SAS options leads to such reduced memory consumption.