DATA Step, Macro, Functions and more

Loading SAS dataset to memory

Accepted Solution Solved
Reply
Frequent Contributor
Posts: 137
Accepted Solution

Loading SAS dataset to memory

Hi I am using the sas option to load SAS tables in memory:

 

sasfile work.ventas load;

 

My dataset has 10 million rows is 2Gb and my RAM is 8Gb, but once loaded in memory I haven’t improve the performance of my proc s and data steps comparing with when the dataset is in disk.

I don’t know if this option (sasfile ... load)  is oriented to improve some types of steps or data management. My steps are proc freq, proc measn and typical data steps to calculate fields

Also perhaps I need a LASR server to work efficiently in memory with SAS. Can anybody help me?

Any advice will be greatly apreciated.


Accepted Solutions
Solution
‎09-30-2017 01:30 PM
Trusted Advisor
Posts: 1,309

Re: Loading SAS dataset to memory

Posted in reply to juanvg1972

As a pure test of reading the dataset from memory, run a DATA _NULL_ from disk and again from the same dataset via sasfile:

 

data _null_;
  set work.ventas;
run;


sasfile work.ventas load;
data _null_;
  set work.ventas;
run;


 

 

and compare the timings of the data _null_ steps,

 

 

View solution in original post


All Replies
Valued Guide
Posts: 570

Re: Loading SAS dataset to memory

Posted in reply to juanvg1972
Respected Advisor
Posts: 4,668

Re: Loading SAS dataset to memory

[ Edited ]
Posted in reply to juanvg1972

@juanvg1972

Proc Means will likely need to sort the data which is done multithreaded but using temporary intermediary data sets on disc (UTILLOC). Having your source table in memory will at best speed up read from source into this process - but depending on the inner workings of Proc Means this read operation isn't the "bottleneck" and though reading from memory won't improve performance at all.

 

Given above: If and when having data in memory improves performance depends on the processing details.

 

There are options like BUFNO which influence performance for read/write operations on disk and it can be worth to investigate if the default values are set optimal for your environment.

 

Here a paper which might be useful:

http://support.sas.com/resources/papers/proceedings09/333-2009.pdf 

 

Solution
‎09-30-2017 01:30 PM
Trusted Advisor
Posts: 1,309

Re: Loading SAS dataset to memory

Posted in reply to juanvg1972

As a pure test of reading the dataset from memory, run a DATA _NULL_ from disk and again from the same dataset via sasfile:

 

data _null_;
  set work.ventas;
run;


sasfile work.ventas load;
data _null_;
  set work.ventas;
run;


 

 

and compare the timings of the data _null_ steps,

 

 

☑ This topic is solved.

Need further help from the community? Please ask a new question.

Discussion stats
  • 3 replies
  • 425 views
  • 3 likes
  • 4 in conversation