topic Re: SAS Visual Analytics - Max size of dataset? in SAS Visual Analytics

SAS Visual Analytics - Max size of dataset?

Filipvdr — Thu, 18 May 2017 09:02:36 GMT

Hello, we recently installed SAS VA.

We are using the autoload folder to load data. This works for our datasets, except a larger dataset of 7GB.

We have 100GB of RAM memory in total. At this point only 5GB is used.

We wanted to load a SAS Dataset of 7GB GB but it could not be loaded.

Any idea or setting we should change?

Thanks,

Filip

Re: SAS Visual Analytics - Max size of dataset?

RW9 — Thu, 18 May 2017 09:09:51 GMT

That sounds like a pretty large amount of data, is it all really needed for some visualisations? Not a VA user myself so don't know the limitations of hand, although there is a page of limitations at:

http://support.sas.com/documentation/cdl/en/vaug/68648/HTML/default/viewer.htm#p1fchf7e8fjsran18qhzftx2xkkd.htm

Me, I would break the data out into relevant blocks, summarise where needed, then work with a lot smaller data.

Re: SAS Visual Analytics - Max size of dataset?

Filipvdr — Thu, 18 May 2017 09:12:27 GMT

@RW9, you're link is pointing to information maps.. we are not using this. We just load SAS Datasets. So your answer is not an answer to my question.

Re: SAS Visual Analytics - Max size of dataset?

JuanS_OCS — Thu, 18 May 2017 10:55:21 GMT

Hello @Filipvdr,

good question there.

On a default SAS installation, you will get a MEMSIZE option of 2G - 8G(GigaBytes), this is the memory size of sas.exe, the process where SAS runs.

The LASR process is like an enhanced version of sas.exe, but it runs with some extra components (as tkgrid) and they are Workspace Server sessions. Therefore it has same characterictics.

If you installed by yourself, probably you forgot or you have no clue about customizations for the SAS VA solution. If a SAS Consultant or an especialized partner installed the server, normally we implement all the best practices on the installation process.

So, and to wrap it up, what you would need is to add the following line to your /LevX/SASApp/sasv9_usermods.cfg, and restart all SAS services and the LASR processes.

-MEMSIZE 0

This will enable your LASR sessions to use unlimited amount of memory, and you will be able to load bigger tables.

PS. For more info, please read this great post from @ErwanGranger : https://communities.sas.com/t5/SAS-Communities-Library/It-s-the-memsize/ta-p/239224

Re: SAS Visual Analytics - Max size of dataset?

SASKiwi — Thu, 18 May 2017 23:51:15 GMT

We load tables up to 20GB in size. If your settings are set correctly the only upper limit is the size of your memory although I wouldn't recommend loading a single table that completely fills your memory.