A note on SAS Visual Analytics 7.1: Data compression

[#1]

As mentioned in a previous article, I’ll post information on new features offered in SAS Visual Analytics 7.1 over the next several days. Here’s an update for the day:

With SAS Visual Analytics 7.1, available Oct. 7, you can work with data sets larger than your available physical memory by compressing them and then loading into SAS LASR.

Will this process open up some possibilities for you?

[MichelleHomes]

I imagine that this could assist many sites being able to compact the size. I'm curious whether are any performance impacts with working with a compressed data source in the SAS LASR server and if there is any benchmarking/testing that can be shared.

[justin_sas]

Hi Michelle,

We agree compression could help a lot to manage your memory consumption. Generally one would expect a tradeoff between storage and responsiveness. However, while we do not have any benchmarks to publish now, anecdotally from our experience we have found around 10-20+ compression depending on the data type, size, and columns types. Often with no significant difference in speed when creating or rendering basic report elements such as bar/line charts, crosstabs, sliders, etc. Some actions such as adding data can actually be faster while actions with some calculations such as in a scatter plot or box plot can be up to 2-3x slower (not bad for 20x memory usage improvement). The exact trade off equation depends on many factors including the make up of your daata, so we encourage you to run your own tests to decide when to take advantage of this new feature.

Cheers,

Justin

[MichelleHomes]

Thanks Justin. Great to hear and I'm impressed with the rates of compression benefits.

As each environment is different with their own hardware, network, data attributes, reports and explorations it is wise to do independent testing however having some guidelines and/or scenarios based on SAS' experiences would be helpful.