We’re smarter together. Learn from this collection of community knowledge and add your expertise.

How does memory management work in SAS?

by Community Manager on ‎01-26-2016 07:57 PM (736 Views)


How does memory management work in SAS?


This question originally appeared on Quora, where the original poster was wondering about SAS memory management and whether there was a garbage collection mechanism.


This answer not technically deep, but I hope it helps those who are getting started with SAS programming and might be looking for how memory works compared to other programming languages.  


For the purposes of this answer, we're going to assume that the question is about the SAS programming language. That is, as a SAS programmer, what do you need to know about how memory is managed as your SAS program is running?


SAS manages the memory for you, so it's similar to Java or .NET in that way. But there isn't a garbage collection mechanism per se. SAS programs consist of a series of "steps" -- the DATA step and PROC steps (for SAS procedures). For the most part, memory is allocated as needed during the running step, then released when the step is complete.


Some SAS procedures can use a lot of memory during the course of their work, and you can run out of memory when working with large data or data sets with large cardinality. For example, PROC FREQ can compute n-way frequencies quickly, but if the variables you're analyzing have lots of distinct values, you can bump into memory limits. When this happens, SAS will issue an ERROR message to the SAS log indicating that there wasn't enough memory to complete the operation.  


Some statistical procedures, such as PROC MIXED or PROC REG and others, can also use a lot of memory for intense computations.  Programmers often have choices for how they approach a statisistical problem, trading off in-memory speed for multiple iterations that are slower, but can work within memory constraints.  The documentation for these procedures will often include a section about "Computational Resources"  to help guide you.  Here's an example about PROC LOGISTIC.


The SAS DATA step also offers some constructs that are inherently memory-dependent. The most popular is the hash object, which allows you to load lots of data values into memory and manipulate them quickly. It's good for building fast data management algorithms, as long as the data values can fit into memory.


The SAS FULLSTIMER system option can show you how much memory a procedure or DATA step uses within your program.  This detailed information can be helpful as you optimize your SAS programs in a constrained environment.


Finally, when we talk about memory management in SAS, we also have to address the WORK area: the temporary scratch space on disk where SAS can write files as intermediate results while SAS procedures run. In practice, most SAS programs that deal with large data sets will run into WORK space limitation before they hit memory constraints. On administered SAS environments, it's important to ensure that SAS programs have access to enough WORK space (ideally with fast I/O performance) to accomplish their work.


For more admin-friendly tips, see this series of blog posts from a SAS performance expert, Margaret Crevar.