We’re smarter together. Learn from this collection of community knowledge and add your expertise.

A note on SAS Visual Analytics 7.1: Data compression

by Community Manager on ‎10-06-2014 02:27 PM - edited on ‎10-05-2015 04:39 PM by Community Manager (1,394 Views)

As mentioned in a previous article, I’ll post information on new features offered in SAS Visual Analytics 7.1 over the next several days. Here’s an update for the day:

 

With SAS Visual Analytics 7.1, available Oct. 7, you can work with data sets larger than your available physical memory by compressing them and then loading into SAS LASR.

 

Will this process open up some possibilities for you?

Comments
by Trusted Advisor
on ‎10-06-2014 07:12 PM

I imagine that this could assist many sites being able to compact the size. I'm curious whether are any performance impacts with working with a compressed data source in the SAS LASR server and if there is any benchmarking/testing that can be shared.

by SAS Employee justin_sas
on ‎10-08-2014 04:34 PM

Hi Michelle,


We agree compression could help a lot to manage your memory consumption. Generally one would expect a tradeoff between storage and responsiveness. However, while we do not have any benchmarks to publish now, anecdotally from our experience we have found around 10-20+ compression depending on the data type, size, and columns types. Often with no significant difference in speed when creating or rendering basic report elements such as bar/line charts, crosstabs, sliders, etc. Some actions such as adding data can actually be faster while actions with some calculations such as in a scatter plot or box plot can be up to 2-3x slower (not bad for 20x memory usage improvement). The exact trade off equation depends on many factors including the make up of your daata, so we encourage you to run your own tests to decide when to take advantage of this new feature.


Cheers,

Justin

by Trusted Advisor
on ‎10-08-2014 05:17 PM

Thanks Justin. Great to hear and I'm impressed with the rates of compression benefits.

As each environment is different with their own hardware, network, data attributes, reports and explorations it is wise to do independent testing however having some guidelines and/or scenarios based on SAS' experiences would be helpful.

Your turn
Sign In!

Want to write an article? Sign in with your profile.


Looking for the Ask the Expert series? Find it in its new home: communities.sas.com/askexpert.