topic DATA step and memory requirements in SAS Procedures

DATA step and memory requirements

ChrisT — Tue, 17 Jan 2012 21:26:27 GMT

Hi,

I have a question regarding the memory usage of the DATA step. In a sample program like:

DATA weight;

input ID $ Week1 Week16;

WeightLoss=Week1-Week16;

datalines;

2477 195 163

2431 220 198

2456 173 155

2412 135 116

;

run;

Where SAS stores the dataset "weight" while this step is running? Does it store it in RAM or in a temporary file in the filesystem? What if much more data is being read in this step?

Thanks,

Chris

DATA step and memory requirements

art297 — Tue, 17 Jan 2012 21:35:24 GMT

I think the short answer would be both, but mostly in a temporary file.

DATA step and memory requirements

ChrisT — Tue, 17 Jan 2012 21:39:32 GMT

Thanks art297.

Is there a way to prevent SAS from doing that and instead instruct it to just keep the dataset in RAM?

This way we avoid the I/O involved in writing temporary data out to the disk.

Chris

DATA step and memory requirements

Doc_Duke — Tue, 17 Jan 2012 22:10:19 GMT

Not really. The data is stored in the WORK directory. WORK is part of a file system in whatever OS you are using. You might be able to store it in RAM if you created a virtual disk by using some of your RAM for that. You could also create the data set as a VIEW; however, it would then do the I/O processing each time you went to use it.

In general, the advantage of SAS is that it can work with arbitrarily large data. The flip side is that it is an I/O hog sometimes.

Doc Muhlbaier

Duke

DATA step and memory requirements

Tom — Tue, 17 Jan 2012 22:37:28 GMT

Why? If you want to do everything in RAM try using PROC IML.

Once you have a dataset you can use the SASFILE command to load it into RAM, but I cannot think of anyway to do it while creating the file.