If you are working with large data, the end goal is to gain some value from it. It may be as simple as a query for sales last quarter or as complex as a creating sales forecast. The task still requires that you write a program to retrieve the data, shape and aggregate it, and then create the desired output. The entire focus for efficiency is about how well we can manipulate and move the data around.
When seeking programming efficiencies, many new programmers and even administrators might think that performance is related only to the hardware. However, powerful servers are only a small part of the equation. If you install a powerful server without making other changes, then you only enable bloated databases and inefficient programs to run lighting quick!
Performance should be approached from several viewpoints as shown in the following figure. By working with each of these environment components you can ensure you are getting the most performance.
Join Ben Murphy and Nick Welke at SAS Global Forum as they present Quickish Performance Techniques for Biggish Data. This paper provides some quickishtechniques Zencos has applied for working with biggish data that made the difference.
... View more