topic Re: MRM dataload job memory issue in Administration and Deployment

MRM dataload job memory issue

woo — Thu, 23 Aug 2018 17:39:53 GMT

I am running sas 9.4 on Linux vm.

I am running into below error when I run dataload job for MRM application.

ERROR: Insufficient resources to perform the operation.
ERROR: The SAS System stopped processing this step because of insufficient memory.

this error pop up when code is trying to Loads tables into LASR and applies RLS permissions, simple data step and set statement (below).

data MRMlasr.MODELREVIEW_lasr;
set MRMdata.MODELREVIEW;
run;

I have set up memsize to max (which is like ~30 gb). however, only 2 gb is available all the time on vm (only have ~30 gb total memory) + buffer/cached available is ~26 gb. while running job, I am keeping eye on system resources (with top cmd) but don't see any SAS session in top 3 top usage processes.

any thoughts/ideas what else can be done? Thanks.

Re: MRM dataload job memory issue

SASKiwi — Thu, 23 Aug 2018 20:14:35 GMT

Please add the FULLSTIMER SAS option to your SAS code and post the log. This will show actual memory usage.

Re: MRM dataload job memory issue

woo — Thu, 23 Aug 2018 21:53:54 GMT

data MRMlasr.MODELREVIEW_lasr;
set MRMdata.MODELREVIEW;
run;

ERROR: Insufficient resources to perform the operation.
ERROR: The SAS System stopped processing this step because of insufficient memory.
NOTE: There were 1597 observations read from the data set MRMDATA.MODELREVIEW.
WARNING: The data set MRMLASR.MODELREVIEW_LASR may be incomplete. When this step was stopped there were 1597 observations and 376 variables.
NOTE: DATA statement used (Total process time):
      real time           4.19 seconds
      user cpu time       0.46 seconds
      system cpu time     0.27 seconds
      memory              16181.96k
      OS Memory           39888.00k
      Step Count                        1746 Switch Count 54
      Page Faults                       0
      Page Reclaims                     4150
      Page Swaps                        0
      Voluntary Context Switches        695
      Involuntary Context Switches      74

Re: MRM dataload job memory issue

SASKiwi — Thu, 23 Aug 2018 22:40:22 GMT

That is strange as your DATA step is only reported using 16 MB of memory. It looks like you are loading a table into LASR memory.

Please check how much free space you have in your LASR library MRMlasr - maybe you have run out of space there.

Re: MRM dataload job memory issue

SimonDawson — Fri, 24 Aug 2018 05:27:13 GMT

That is the memory usage of the step. The step only really needs a PDV allocated. The ERROR is coming from the library you are trying to write to.

You are more than likely hitting the memory limit on the LASR server. If its SMP the SAS session MEMSIZE option is the cap. In an MMP configuration there are some options for restricting the memory allocator via an environment vairable and there is also some options you can add to the proc lasr that starts the server.

http://support.sas.com/documentation/cdl/en/inmsref/68736/HTML/default/viewer.htm#n18hbm82nhmooan1hxz5fhzu68w0.htm

Re: MRM dataload job memory issue

woo — Fri, 24 Aug 2018 19:13:41 GMT

thanks SASKivi, Simon.

mrmlasr is point to where we load lasr tables and we have enough space down there, 100+ gb.

I have memsize setup at max.

@Simon -> I don't find "resource.setting", I can't be root to find that file,

Re: MRM dataload job memory issue

woo — Fri, 24 Aug 2018 21:52:52 GMT

I am also removing older sas datasets from different libraries before running new job making sure no older datasets sitting out there.

also, mrmlasr is a SASIOLA library. so if I highlight mrmlasr library from sas mc from "library manager" it shows lot of tables. do I need to remove/delete those tables as well? Thanks...

Re: MRM dataload job memory issue

SASKiwi — Fri, 24 Aug 2018 22:09:48 GMT

Your MRMLASR library is limited to the amount of memory allocated to it. You need to delete tables no longer required in this library to free up memory. I never used MRM, but I would hope it has something like the SAS VA Administrator web page. This is a good place to check your MRMLASR tables as it tells you how much space they take up and you can also unload tables there.

Re: MRM dataload job memory issue

woo — Tue, 28 Aug 2018 15:52:44 GMT

Thank you SASKIWI.

So mrmlasr is lasr library and data provider library (base library) is different for that "mrmlasr" lasr library.

I cleaned up data provider library (base library) at os level by removing physical data created from dataload job. trying to figure if any metadata table definition tied up with "mrmlasr" lasr library needed for dataload job and if I can remove other metadata table definition as well.

Re: MRM dataload job memory issue

woo — Tue, 28 Aug 2018 16:02:39 GMT

also, from "sas visual analytics server admin" tool I can see most of tables (for "mrmlasr" lasr library) are in "unloaded" status so I believe there is not much to delete there. however, there are 100 + tables (metadata definition) for mrmlasr library are out there.