DATA Step, Macro, Functions and more

Understanding Efficiency Trade-offs?

Accepted Solution Solved
Reply
Contributor
Posts: 35
Accepted Solution

Understanding Efficiency Trade-offs?

Hi Everyone,

Can anyone please explain the Efficiency Trade-offs in SAS. I am having hard time

understanding the above concepts.Following are two questions:

1)How decrease in the size of a SAS Dataset can result in an increase in CPU Usage?

2)How decrease in the number of I/O operations comes at the expense of Increased

memory usage?

Thank you,

John


Accepted Solutions
Solution
‎04-17-2012 02:46 PM
Trusted Advisor
Posts: 1,301

Re: Understanding Efficiency Trade-offs?

I would definitely recommend reviewing the papers Cynthia has provided as they will provide a lot more context and information than the following.

1) This is most likely a reference to data set compression, using the COMPRESS option, this decreases the size of the dataset on disk and in turn the number of I/O operations required to access it.  This increases CPU Usage because when reading or writing compressed data there is the extra step to perform the decompression or compression of said data.  I do not believe there is a real impact to memory with this.

2) I will take a wild guess here and say you are referring to the SORTSIZE option in PROC SORT, but similar memory size options will relate somewhat.  By increasing the allowable memory to a sort you can reduce the number of I/O operations performed of the utility files because larger portions of the file can be handled in memory at a time.  I/O is typically the most time-expensive operation.  SASFILE can also help reduce I/O operations for steps that involve reading files multiple times, as this will place the contents of the file into memory.

FULLSTIMER option is good to use and will help you see where processes are spending their time and what the costs of various tweaks are:

http://support.sas.com/rnd/scalability/tools/fullstim/index.html

View solution in original post


All Replies
SAS Super FREQ
Posts: 8,865

Re: Understanding Efficiency Trade-offs?


Hi:

  Efficiency is always a trade-off -- decreasing one resource can, and frequently does, increase another resource. This is a good place to start in reading about efficiency techniques:

http://www2.sas.com/proceedings/forum2007/042-2007.pdf

  And a Google search on SAS Efficiency reveals other papers, like these:

http://www.nesug.org/Proceedings/nesug11/ds/ds02.pdf

http://www.mwsug.org/proceedings/2009/how/MWSUG-2009-H06.pdf

  I'm sure you can find more.

cynthia

Solution
‎04-17-2012 02:46 PM
Trusted Advisor
Posts: 1,301

Re: Understanding Efficiency Trade-offs?

I would definitely recommend reviewing the papers Cynthia has provided as they will provide a lot more context and information than the following.

1) This is most likely a reference to data set compression, using the COMPRESS option, this decreases the size of the dataset on disk and in turn the number of I/O operations required to access it.  This increases CPU Usage because when reading or writing compressed data there is the extra step to perform the decompression or compression of said data.  I do not believe there is a real impact to memory with this.

2) I will take a wild guess here and say you are referring to the SORTSIZE option in PROC SORT, but similar memory size options will relate somewhat.  By increasing the allowable memory to a sort you can reduce the number of I/O operations performed of the utility files because larger portions of the file can be handled in memory at a time.  I/O is typically the most time-expensive operation.  SASFILE can also help reduce I/O operations for steps that involve reading files multiple times, as this will place the contents of the file into memory.

FULLSTIMER option is good to use and will help you see where processes are spending their time and what the costs of various tweaks are:

http://support.sas.com/rnd/scalability/tools/fullstim/index.html

🔒 This topic is solved and locked.

Need further help from the community? Please ask a new question.

Discussion stats
  • 2 replies
  • 267 views
  • 0 likes
  • 3 in conversation