DATA Step, Macro, Functions and more

what is the most efficient way to access and Manupluate 200 Million Records from a table (s)

Reply
Occasional Contributor
Posts: 9

what is the most efficient way to access and Manupluate 200 Million Records from a table (s)

Dears;

 

our customer has a growning anlytics DBase. The current size for one of its tables is almost 200,000,000 Records. my team tried to do a simple query on the table but it spent more than 30 minutes with no results.

 

can anyone suggest the best approach to access it? Partitioning? cashing?...

 

 

thank you in advance.

Super User
Posts: 3,102

Re: what is the most efficient way to access and Manupluate 200 Million Records from a table (s)

[ Edited ]

I'm assuming you are talking about a SAS table here since this is a SAS community. Please provide an example of the slow query including SAS log statistics. 200 million rows is not that large. How many columns does it have and what is the physical size of the table in gigabytes?

 

If the table has a lot of columns is it compressed or not? Often simply compressing tables can dramatically improve query performance. There are many other possibilities for speeding up processing but until you provide more information, it is difficult to offer the right advice.

 

You can analyse the performance of your table queries by following this advice:

 

http://support.sas.com/rnd/scalability/papers/solve_perf.pdf

 

Ask a Question
Discussion stats
  • 1 reply
  • 198 views
  • 1 like
  • 2 in conversation