Determining the correct MEMSIZE value - server with 256GB RAM and about 60 users and large datasets

[gothaggis]

Originally I had this value set to 4GB, however many users programs could not execute - so then i upped it to 22GB and several users programs still could not execute.  As a test I upped it to 64GB, and all programs could execute, but this scares me as I don't think this value should be that high.  The users do work with quite large datasets - but is there anything they can do on their end to limit the amount of memory their programs require?  What would be the ideal MEMSIZE setting for a server like this - or does it totally depend on the size of the datasets users are working with?

[Kurt_Bremser]

Some procedures can be made less (much less) memory-consuming by sorting datasets and using by instead of class, as an example.

It all depends on what kind of analysis is done.

[JuanS_OCS]

I suggest to limit the values and let the programs to fail. If any program needs more, let the request to escalate snd the to csn set a priviledged sas session with additional memory if needed.

What the users could do us to use the fullstimer option, log their executions and then systematically identify the data steps or procedures from each sas job that actually deviate from normal and allowed executions. Once the unitary execution that deviates os identified, you snd they csn work on its optimization, easily.

Your concern about the memsize is well founded. What you csn do as well is to identify the top number of concurrent sas sessions, as logged in your system, independently from those 60 users: in one hand, not all will log at the same time, in other hand, some of them might create more than one session.

The fact is this: 256 GB of RAM, can hold no more than 4 concurrent SAS sessions, in RAM, that use completely the 64 GB of MEMSIZE. After this limit, the next concurrent sessions will be able to execute as well but it will cause your system to continously paginate and use the swap/system page file. Meaning: it will be much slower, it will unnecesarily erosionate your physical storage due to pagination, making the lifespan of that storage volume shorter and, what is the most concerning of all: once the swap/page file is overloaded, your operating system will, most likely, crash completely.

I hope it helps.

[SASKiwi]

It is worth remembering that the default value of MEMSIZE at installation time as set by SAS is around 3 or 4GB. This value is a good compromise between allowing most SAS jobs to run without memory problems, while still leaving enough for other SAS jobs to run at the same time.

Greatly increasing this value for all SAS users is not recommended for the reasons explained by @JuanS_OCS. Small increases up to say 8GB are probably OK but that depends on the number of simultaneous SAS sessions compared to server physical memory.

[SimonDawson]

I had a conversation with @ChrisNZ a while back where we discussed MEMSIZE and related tunables at length. Here is a snippet of that conversation where I shared some of my thoughts on the topic at hand.

With modern machines I'd say this (memory allocator limits and related tunables) is mattering less and less. When your computer had 32Kb of memory you'd invest in the software to make smart decisions with memory to get your problem solved. With so little memory in the machines of the time it was easy to have your working set of data be orders of magnitude larger than the available memory you could physically fit in a machine. On top of this the ISA for most CPUs only supported addressing up to a maximum of 4GB of virtual memory. When this options was added into SAS I'm sure they didn't even consider that one day we'd need to have a new set of instructions for CPUs and entirely new CPUs to address more than 4GB of memory. For a lot of computing problems now if you don't have enough memory you just buy more (or cluster machines i.e. the SAS LASR Analytics Server). This is also likely reflected in the SAS procedures too. I'm sure its only the older procedures that have the smarts to dynamically pick algorithms based on the actual amount of memory in the system.

With the Linux kernels control groups feature getting quite mature now this is the route I'd likely take to solve the issue of users taking all the memory on a box if the issue arose. The MEMSIZE tunable inside SAS is great but it has got some obvious short comings. It only track memory allocated by the SAS session's memory management subsystem. Memory allocated by third party software (like database client drivers) isn't accounted for. Another problem is that a user can simple spawn multiple processes and easily consume more memory than you expected them to take. With control groups you can set resource limits on all of the users processes as a whole. This includes memory allocated by their SAS processes, loaded third party libraries (e.g. database client libraries), even processes spawned by their SAS session which would get a whole new address space (eg. bulkloaders, X statements, etc).

For @gothaggis the simplest answer I'd give you is to set it high enough that users don't complain, re-evaluate the situation if you see the oom-killer invoked and the box is running out of memory.

[JuanS_OCS]

Hi @SimonDawson , true!

Do you have any document with some recommendations or practices based on experience, as used for the SAS Software?

That would be really interesting!

Thanks in advance,

Kind regards,

Juan

[SimonDawson]

Sorry @JuanS_OCS this is only something I've given a few thoughts to on my own time. I used Linux control groups outside of SAS in my own personal projects so I've got some experience with it for other things. It would be neat to do some research into how to leverage the technology the kernel provides for a SAS deployment. Perhaps when I get a few spare hours I might try experiment with it.

[JuanS_OCS]

It would be worth a (great) SAS Paper for a Forum, @SimonDawson 

Kudos for you.