04-09-2017 04:24 PM
We have Visual Analytics 7.3 ( installed in a distributed environment -4 worker nodes with 128GB RAM each.(no co-located data store) I am loading data from Hive(separate Hadoop cluster w/o SAS compoents) into LASR. My users want to search raw data sets which are aproximately 200GB+. I have no problem loading smaller sets which are as large as 80GB into LASR, but the 200GB+ fail to load. When they apply the filters to get to the data they need, it should be less than 1% of the 200GB. Pulling it all into the LASR memory structures seem to be very cumbersome for this use case. I am thinking a different SAS tool would have better served their needs. I am thinking Data Miner, Web Reports or SAS/ACCESS Interface to Hadoop might map well. Advice is appreciated.
04-09-2017 05:30 PM
I would suggest that running SQL queries against the data in Hadoop would be a far better option. LASR servers should be for reporting and exploring using the VA front-end only.