topic Re: Loading SAS dataset to memory in SAS Programming

Loading SAS dataset to memory

juanvg1972 — Fri, 29 Sep 2017 23:13:21 GMT

Hi I am using the sas option to load SAS tables in memory:

sasfile work.ventas load;

My dataset has 10 million rows is 2Gb and my RAM is 8Gb, but once loaded in memory I haven’t improve the performance of my proc s and data steps comparing with when the dataset is in disk.

I don’t know if this option (sasfile ... load) is oriented to improve some types of steps or data management. My steps are proc freq, proc measn and typical data steps to calculate fields

Also perhaps I need a LASR server to work efficiently in memory with SAS. Can anybody help me?

Any advice will be greatly apreciated.

Re: Loading SAS dataset to memory

ChrisBrooks — Fri, 29 Sep 2017 23:29:40 GMT

Have you tried increasing the value of the Bufno system option? http://support.sas.com/documentation/cdl/en/hostwin/69955/HTML/default/viewer.htm#p148jwukxcnmmin0z662qex78l75.htm

Re: Loading SAS dataset to memory

Patrick — Sat, 30 Sep 2017 00:20:53 GMT

@juanvg1972

Proc Means will likely need to sort the data which is done multithreaded but using temporary intermediary data sets on disc (UTILLOC). Having your source table in memory will at best speed up read from source into this process - but depending on the inner workings of Proc Means this read operation isn't the "bottleneck" and though reading from memory won't improve performance at all.

Given above: If and when having data in memory improves performance depends on the processing details.

There are options like BUFNO which influence performance for read/write operations on disk and it can be worth to investigate if the default values are set optimal for your environment.

Here a paper which might be useful:

http://support.sas.com/resources/papers/proceedings09/333-2009.pdf

Re: Loading SAS dataset to memory

mkeintz — Sat, 30 Sep 2017 05:14:03 GMT

As a pure test of reading the dataset from memory, run a DATA _NULL_ from disk and again from the same dataset via sasfile:

data _null_;
  set work.ventas;
run;


sasfile work.ventas load;
data _null_;
  set work.ventas;
run;

and compare the timings of the data _null_ steps,