Compressing Tables on LASR - Does it uncompress the entire table

[udaypatel]

Hello,

We are running a compressing on two Large tables (~10million and ~100million rows). Using the compress proc we reduce their physical size by a ratio of 7.3:1.

Performance of the report viewer seems to degrade as the number of rows is increased. Does this mean the report attempts to uncompress the entire table or just the blocks that are required depending on the variables that are selected?

Thanks in advance!

[udaypatel]

I have found this on reference guide:

Performance Considerations

Compression exchanges less memory use for more CPU use. It slows down any request that processes the data. An in-memory table consists of blocks of rows. When the server works with a compressed table, the blocks of rows must be uncompressed before the server can work with the variables. In some cases, a request can require five times longer to run with a compressed table rather than an uncompressed table.

For example, if you want to summarize two variables in a table that has 100 variables, all 100 columns must be uncompressed in order to locate the data for the two variables of interest. If you specify a WHERE clause, then the server must uncompress the data before the WHERE clause can be applied. Like the example where only two of 100 variables are used, if the WHERE clause is very restrictive, then there is a substantial performance penalty to filter out most of the rows.

Working with SASHDAT tables that are loaded from HDFS is the most memory-efficient way to use the server. Using compressed SASHDAT tables preserves the memory efficiencies, but still incurs the performance penalty of uncompressing the rows as the server operates on each row..

I guess it is true - all blocks of that variable must be uncompressed first 😞