Exploring, predicting and reporting with SAS Visual Analytics and SAS Visual Statistics

Visual Analytics the right tool to investigate/search Large Data Sets stored in Hive/Hadoop

Reply
Regular Learner
Posts: 1

Visual Analytics the right tool to investigate/search Large Data Sets stored in Hive/Hadoop

We have Visual Analytics 7.3 ( installed in a distributed environment -4 worker nodes with 128GB RAM each.(no co-located data store)  I am loading data from Hive(separate Hadoop cluster w/o SAS compoents) into LASR.  My users want to search raw data sets which are aproximately 200GB+.  I have no problem loading smaller sets which are as large as 80GB into LASR, but the 200GB+ fail to load.  When they apply the filters to get to the data they need, it should be less than 1% of the 200GB.  Pulling it all into the LASR memory structures seem to be very cumbersome for this use case.   I am thinking a different SAS tool would have better served their needs.  I am thinking Data Miner, Web Reports or SAS/ACCESS Interface to Hadoop might map well.  Advice is appreciated.

Super User
Posts: 3,233

Re: Visual Analytics the right tool to investigate/search Large Data Sets stored in Hive/Hadoop

I would suggest that running SQL queries against the data in Hadoop would be a far better option. LASR servers should be for  reporting and exploring using the VA front-end only.

Ask a Question
Discussion stats
  • 1 reply
  • 145 views
  • 1 like
  • 2 in conversation