Help using Base SAS procedures

Speeding up SAS

Reply
Frequent Contributor
Posts: 117

Speeding up SAS

Hi guys,

I have some fairly large data sets that are created on a monthly basis. I was wondering if you have any tips on data compression techniqes to speed up the time SAS takes to create these data sets?

Thanks for the tips!

Frequent Contributor
Posts: 110

Speeding up SAS

Instead of using proc sql by libname reference go through Oracle or ODBMS pass through. try to use dbkey in proc sql. It drastically increases the join times.

Frequent Contributor
Posts: 110

Speeding up SAS

just paste some code here. there are legends they help you tune up your code.

Super User
Posts: 5,079

Speeding up SAS

vomer,

You're asking two different questions.  Compression affects disk space usage.  As a general rule, this does not speed up programs but rather slows them down slightly.  What are you trying to accomplish?

Frequent Contributor
Posts: 117

Speeding up SAS

Mainly looking to speed up the code run time. Wondering if thewe are any tricks that everyone here has used to speed things up in general.

Super User
Super User
Posts: 6,498

Speeding up SAS

Summarize as early in the process as possible.

PROC Star
Posts: 7,360

Speeding up SAS

Like others have said, seeing what your data and data needs are, as well as environment, would have to be known in order for anyone to suggest valid possibilities.

Tom mentioned summarizing as early as possible.  I agree, but would even back up a step before that.  Only import and/or keep the data that are really necessary.

Trusted Advisor
Posts: 1,300

Speeding up SAS

Astounding,

Since the slower computing operation is typically I/O compression often does speed up programs by allowing the disk to push the data faster at the cost of increased CPU usage.  It all depends where your bottlenecking in performance.

vomer,

Please provide additional information about how these files are created currently on a monthly basis and any system information you know or feel comfortable sharing, especially OS, available storage types, cpu counts, ram, etc..

What is the nature of the data, does it have natural segmentation for the analysis you intend to do with it?  Why do you feel this program is running slow?  What is it's current performance metrics and where would you want it to be?

Increasing performance is a very large question to ask so vaguely.

Super Contributor
Posts: 356

Speeding up SAS

In My experience compressing has actually speed up the process, it really depends on where the bottle neck is... Ie if you have slow disk, writing/reading as little as possible helps...

Barry

Super User
Posts: 5,255

Re: Speeding up SAS

If we look at compression as such, if your data is mainly character, try the ordinary CHAR method.

If you have lots of numerical data, try BINARY.

If you have lots of numerical integers, you can try to specify other length than the default 8.

As Barry says, it deponds on your system and data, if compression will help processing time. The penalty for compression is more CPU cycles.

My experience is that you need at least 50% compression to gain shorter processing time.

Hoe long is your run time anyway? Do you have indexes defined?

Depending on the requirements, some re-modelling of the data can sometimes help.

Data never sleeps
Super User
Posts: 10,483

Re: Speeding up SAS

If I don't want to babysit the program while running I'd be tempted to schedule it to run while I'm out of the office, say overnight if possible. Then everything is done when I come in the morning.

Ask a Question
Discussion stats
  • 10 replies
  • 726 views
  • 2 likes
  • 9 in conversation