topic Re: Understanding Efficiency Trade-offs? in SAS Programming

Understanding Efficiency Trade-offs?

john83 — Tue, 17 Apr 2012 15:49:05 GMT

Hi Everyone,

Can anyone please explain the Efficiency Trade-offs in SAS. I am having hard time

understanding the above concepts.Following are two questions:

1)How decrease in the size of a SAS Dataset can result in an increase in CPU Usage?

2)How decrease in the number of I/O operations comes at the expense of Increased

memory usage?

Thank you,

John

Re: Understanding Efficiency Trade-offs?

Cynthia_sas — Tue, 17 Apr 2012 16:04:01 GMT

Hi:

Efficiency is always a trade-off -- decreasing one resource can, and frequently does, increase another resource. This is a good place to start in reading about efficiency techniques:

http://www2.sas.com/proceedings/forum2007/042-2007.pdf

And a Google search on SAS Efficiency reveals other papers, like these:

http://www.nesug.org/Proceedings/nesug11/ds/ds02.pdf

http://www.mwsug.org/proceedings/2009/how/MWSUG-2009-H06.pdf

I'm sure you can find more.

cynthia

Re: Understanding Efficiency Trade-offs?

FriedEgg — Tue, 17 Apr 2012 18:46:36 GMT

I would definitely recommend reviewing the papers Cynthia has provided as they will provide a lot more context and information than the following.

1) This is most likely a reference to data set compression, using the COMPRESS option, this decreases the size of the dataset on disk and in turn the number of I/O operations required to access it. This increases CPU Usage because when reading or writing compressed data there is the extra step to perform the decompression or compression of said data. I do not believe there is a real impact to memory with this.

2) I will take a wild guess here and say you are referring to the SORTSIZE option in PROC SORT, but similar memory size options will relate somewhat. By increasing the allowable memory to a sort you can reduce the number of I/O operations performed of the utility files because larger portions of the file can be handled in memory at a time. I/O is typically the most time-expensive operation. SASFILE can also help reduce I/O operations for steps that involve reading files multiple times, as this will place the contents of the file into memory.

FULLSTIMER option is good to use and will help you see where processes are spending their time and what the costs of various tweaks are:

http://support.sas.com/rnd/scalability/tools/fullstim/index.html