topic SAS VA Autoload loading large table error in Administration and Deployment

SAS VA Autoload loading large table error

woo — Wed, 24 Nov 2021 19:10:45 GMT

We are using SAS VA 7.5 (non-distributed) and have autoload in place to load the table on Linux. Smaller tables are loading fine vs large tables having "Insufficient resources to perform the operation" + "sas system stopped processing this step because of insufficient memory" error.

SAS VA Linux machine has total of 94 gb RAM

workspaceserver_usermods.sh for SASAppVA server has

USERMODS_OPTIONS="-memsize 0 -cpucount ACTUAL"

va.AuditingEnabled is "false" for Visual Analytics 7.5 definition underneath "SAS Application Infrastructure" Advance settings.

From autoload job log,

When it complains about insufficient resource for ~4 gb table, details is as below,

real time 5.71 sec

user cpu time 1.51 sec

system cpu time .59 sec

memory 6266.09k

OS memory 20648.00k

page fault 0

page reclaims 2903

page swaps 0

Vol. context switches 1133

Invol. context switches 1

block input operations 0

block output operations 240

Re: SAS VA Autoload loading large table error

SASKiwi — Wed, 24 Nov 2021 19:49:41 GMT

4GB isn't big at all. I suggest you raise a Tech Support track is this is likely to need troubleshooting.

Edit: Check VA Administrator web page - Servers tab to confirm how much memory you have used. Is it getting close to the 94GB limit.

Re: SAS VA Autoload loading large table error

woo — Thu, 25 Nov 2021 17:27:54 GMT

Thank you SASKiwi.

There is only ~700 MB used i see for in-memory tables (@ Administrator tab -> LASR -> LASR Tables) with green sign, metadata location: /shared data/sas visual analytics/Public/LASR

There are few tables with red status where size doesn't shows up and metadata location for those tables with red status are,

/shared data/sas visual analytics/Autoload/EVDMLA

/shared data/sas visual analytics/Samples

Re: SAS VA Autoload loading large table error

woo — Thu, 25 Nov 2021 17:34:25 GMT

However, there are lot of tables under "/Shared Data" metadata location. Do we need to clean those up? Thank you.

Re: SAS VA Autoload loading large table error

SASKiwi — Thu, 25 Nov 2021 19:05:25 GMT

If there are LASR tables that you no longer use then its a good idea to remove them. I don't think it will fix your problem though. We also use VA 7.5 and have no problem loading tables bigger than yours. Using the Servers tab is the easiest way to see how much LASR memory is used:

Re: SAS VA Autoload loading large table error

woo — Fri, 26 Nov 2021 02:13:41 GMT

Thank you. Do you have extra memsize or any other autoload specific configuraiton for like any autoload.cfg or runsas.sh file OR all settings are being picked by default sasapp config files?

Re: SAS VA Autoload loading large table error

SASKiwi — Fri, 26 Nov 2021 23:56:23 GMT

I'm pretty sure we are using mostly default settings. Have you tried loading the same table using SAS code and do you get the same error?

To find out what is causing this will require some diagnosing and this is best progressed with SAS Tech Support.

Re: SAS VA Autoload loading large table error

woo — Sun, 28 Nov 2021 03:12:51 GMT

Thank you SASKiwi. We will open a track possibly.

Also, just fyi - we have same error loading table using SAS code. will share resolution once have it.