Visual Analytics the right tool to investigate/search Large Data Sets stored in Hive/Hadoop

racrown — Sun, 09 Apr 2017 20:24:21 GMT

We have Visual Analytics 7.3 ( installed in a distributed environment -4 worker nodes with 128GB RAM each.(no co-located data store) I am loading data from Hive(separate Hadoop cluster w/o SAS compoents) into LASR. My users want to search raw data sets which are aproximately 200GB+. I have no problem loading smaller sets which are as large as 80GB into LASR, but the 200GB+ fail to load. When they apply the filters to get to the data they need, it should be less than 1% of the 200GB. Pulling it all into the LASR memory structures seem to be very cumbersome for this use case. I am thinking a different SAS tool would have better served their needs. I am thinking Data Miner, Web Reports or SAS/ACCESS Interface to Hadoop might map well. Advice is appreciated.

Re: Visual Analytics the right tool to investigate/search Large Data Sets stored in Hive/Hadoop

SASKiwi — Sun, 09 Apr 2017 21:30:32 GMT

I would suggest that running SQL queries against the data in Hadoop would be a far better option. LASR servers should be for reporting and exploring using the VA front-end only.

topic Re: Visual Analytics the right tool to investigate/search Large Data Sets stored in Hive/Hadoop in SAS Visual Analytics

Visual Analytics the right tool to investigate/search Large Data Sets stored in Hive/Hadoop

Re: Visual Analytics the right tool to investigate/search Large Data Sets stored in Hive/Hadoop