BookmarkSubscribeRSS Feed

It's the memsize!

Started ‎12-15-2015 by
Modified ‎12-14-2015 by
Views 9,099

So, say you are working on VA environment with a Non-Distributed LASR server. Everything is working fine.
Your LASR server is started, and you have loaded some tables in it.
You verified that you can access them with the VA applications.
But at some point, as you keep adding more and more tables to the LASR server, it suddenly fails.
And it tells you:

ERROR: Insufficient resources to perform the operation.
ERROR: The SAS System stopped processing this step because of insufficient memory

At least we have a very descriptive error message, which might even be how you found this article in the first place. 
At this stage, I believe your reaction might be to think something along the lines of:
"I have 256 GB of RAM on that server. I am pretty sure I have not used all of it yet.".
You may even run something like the "free -m" command (on Linux) to prove to yourself that you still have space in memory: 

             total       used       free     shared    buffers     cached
Mem:        258287      84336     173951          0        969      68923
-/+ buffers/cache:      14444     243843
Swap:        16383          0      16383

In situations such as these, the culprit, as the post tilte indicates, is likely going to be the Memsize value.

In Non-Distributed VA environments, the LASR server is a single SAS process.
It will either be a SAS Workspace server (if started from the VA Administrator Interface), or a batch server (if scheduled through the Schedule Manager).

Although this SAS process’ job is to hold the LASR server (in a Non-Distributed LASR environment), it still has to abide by the rules of SAS Foundation.
Which means that if your effective memsize for that process is around 2GB (a value I have seen often), when you try to add a dataset that would put this process over the 2 GB threshold, you will encounter that error.

 

Determining the memsize

Looking at the sasv9.cfg files has the disadvantage of requiring you to follow the breadcrumb trail of many files.
The following method is fairly easy way to check what is your effective memsize.

  1. Locate the WorkspaceServer.sh script and move into that folder:
    cd /config/Lev1/SASApp/WorkspaceServer/
  2. Execute the script without bringing the GUI up:
    ./WorkspaceServer.sh -nodms
  3. In the prompt that appears (1?) type 
    proc options group=memory; run;
  4. and press enter. You will see something like:
    Group=MEMORY
     SORTSIZE=1073741824
                       Specifies the amount of memory that is available to the
                       SORT procedure.
     SUMSIZE=0         Specifies a limit on the amount of memory that is
                       available for data summarization procedures when class
                       variables are active.
     MAXMEMQUERY=268435456
                       For certain procedures, specifies the maximum amount of
                       memory that can be allocated per request.
     LOADMEMSIZE=0     Specifies a suggested amount of memory that is needed for
                       executable programs loaded by SAS.
     MEMSIZE=15809437440
                       Specifies the limit on the amount of virtual memory that
                       can be used during a SAS session.
     REALMEMSIZE=0     Specifies the amount of real memory SAS can expect to
                       allocate.
    NOTE: PROCEDURE OPTIONS used (Total process time):
          real time           0.00 seconds
          cpu time            0.00 seconds
  5. In the SAS prompt (2?), type
    endsas;
  6. Now, copy the reported value of the memsize (15809437440 in my case) and google the following text (with your value):
    15809437440 bytes in GB
  7. Google tells me that this is about 14.7237 GB. 

Since I ran that on a VM that has about 16 GB of Memory, that value is ok for me.
If this were a machine that meets the minimum requirements for VA Non-Distributed, I would expect this value to be closer to 64GB.
If however, your value is closer to 2 GB, that is very likely your issue.

If it turns out that your memsize value is too low, the first place to look is in
……./config/Lev1/SASApp/WorkspaceServer/WorkspaceServer_usermods.sh
This file, in VA environments should contain:

USERMODS_OPTIONS="-memsize 0 -cpucount ACTUAL "

After having modified the required files, re-execute the previous test, to make sure these changes were enough.

Once your reported memsize is finally what it should be, remember to restart your LASR server!
Then you should be able to load the amount of data that you need to load.  

Remember that if you are using the Batch Server to start your LASR server, instead of the workspace, it’s that one that you have to check and potentially modify.

For example, in my environment, running the proc options in the SAS Batch Server (/opt/sas/config/Lev1/SASApp/BatchServer/sasbatch.sh -nodms) yields the following result: 

Group=MEMORY
MEMSIZE=2147483648
Specifies the limit on the amount of virtual memory that
can be used during a SAS session.

As you can see, the current Memsize of my batch server is only around 2.1 GB, which may not be enough for LASR. 

 

I hope this article is useful to you. 

Comments

Thanks, it works very good now

Gaétan

Just simply make set memsize 0 under sas foundation, workspace and batch server v9 files, correct?

Version history
Last update:
‎12-14-2015 01:53 PM
Updated by:
Contributors

sas-innovate-2024.png

Join us for SAS Innovate April 16-19 at the Aria in Las Vegas. Bring the team and save big with our group pricing for a limited time only.

Pre-conference courses and tutorials are filling up fast and are always a sellout. Register today to reserve your seat.

 

Register now!

Free course: Data Literacy Essentials

Data Literacy is for all, even absolute beginners. Jump on board with this free e-learning  and boost your career prospects.

Get Started

Article Tags