BookmarkSubscribeRSS Feed
🔒 This topic is solved and locked. Need further help from the community? Please sign in and ask a new question.
john83
Calcite | Level 5

Hi Everyone,

Can anyone please explain the Efficiency Trade-offs in SAS. I am having hard time

understanding the above concepts.Following are two questions:

1)How decrease in the size of a SAS Dataset can result in an increase in CPU Usage?

2)How decrease in the number of I/O operations comes at the expense of Increased

memory usage?

Thank you,

John

1 ACCEPTED SOLUTION

Accepted Solutions
FriedEgg
SAS Employee

I would definitely recommend reviewing the papers Cynthia has provided as they will provide a lot more context and information than the following.

1) This is most likely a reference to data set compression, using the COMPRESS option, this decreases the size of the dataset on disk and in turn the number of I/O operations required to access it.  This increases CPU Usage because when reading or writing compressed data there is the extra step to perform the decompression or compression of said data.  I do not believe there is a real impact to memory with this.

2) I will take a wild guess here and say you are referring to the SORTSIZE option in PROC SORT, but similar memory size options will relate somewhat.  By increasing the allowable memory to a sort you can reduce the number of I/O operations performed of the utility files because larger portions of the file can be handled in memory at a time.  I/O is typically the most time-expensive operation.  SASFILE can also help reduce I/O operations for steps that involve reading files multiple times, as this will place the contents of the file into memory.

FULLSTIMER option is good to use and will help you see where processes are spending their time and what the costs of various tweaks are:

http://support.sas.com/rnd/scalability/tools/fullstim/index.html

View solution in original post

2 REPLIES 2
Cynthia_sas
SAS Super FREQ


Hi:

  Efficiency is always a trade-off -- decreasing one resource can, and frequently does, increase another resource. This is a good place to start in reading about efficiency techniques:

http://www2.sas.com/proceedings/forum2007/042-2007.pdf

  And a Google search on SAS Efficiency reveals other papers, like these:

http://www.nesug.org/Proceedings/nesug11/ds/ds02.pdf

http://www.mwsug.org/proceedings/2009/how/MWSUG-2009-H06.pdf

  I'm sure you can find more.

cynthia

FriedEgg
SAS Employee

I would definitely recommend reviewing the papers Cynthia has provided as they will provide a lot more context and information than the following.

1) This is most likely a reference to data set compression, using the COMPRESS option, this decreases the size of the dataset on disk and in turn the number of I/O operations required to access it.  This increases CPU Usage because when reading or writing compressed data there is the extra step to perform the decompression or compression of said data.  I do not believe there is a real impact to memory with this.

2) I will take a wild guess here and say you are referring to the SORTSIZE option in PROC SORT, but similar memory size options will relate somewhat.  By increasing the allowable memory to a sort you can reduce the number of I/O operations performed of the utility files because larger portions of the file can be handled in memory at a time.  I/O is typically the most time-expensive operation.  SASFILE can also help reduce I/O operations for steps that involve reading files multiple times, as this will place the contents of the file into memory.

FULLSTIMER option is good to use and will help you see where processes are spending their time and what the costs of various tweaks are:

http://support.sas.com/rnd/scalability/tools/fullstim/index.html

sas-innovate-2024.png

Don't miss out on SAS Innovate - Register now for the FREE Livestream!

Can't make it to Vegas? No problem! Watch our general sessions LIVE or on-demand starting April 17th. Hear from SAS execs, best-selling author Adam Grant, Hot Ones host Sean Evans, top tech journalist Kara Swisher, AI expert Cassie Kozyrkov, and the mind-blowing dance crew iLuminate! Plus, get access to over 20 breakout sessions.

 

Register now!

How to Concatenate Values

Learn how use the CAT functions in SAS to join values from multiple variables into a single value.

Find more tutorials on the SAS Users YouTube channel.

Click image to register for webinarClick image to register for webinar

Classroom Training Available!

Select SAS Training centers are offering in-person courses. View upcoming courses for:

View all other training opportunities.

Discussion stats
  • 2 replies
  • 1015 views
  • 0 likes
  • 3 in conversation