topic Re: Limit memory available to R (through PROC IML) in Administration and Deployment

Limit memory available to R (through PROC IML)

snoopy369 — Mon, 02 Aug 2021 18:35:17 GMT

My server went down today, and it looks like it was due to a user running the server out of memory via R, as I see in the notes:

Windows successfully diagnosed a low virtual memory condition. The following programs consumed the most virtual memory: tkrproxy64.exe (6600) consumed 107340787712 bytes

This is the process that SAS creates to manage the R interface. That's around 100GB of memory!

Obviously the R interface isn't configured to respect MEMSIZE, which is understandable. How can I limit the memory it's allowed? Can I configure the command SAS uses to start the process (using the --max-mem-size flag) or is there a correct configuration file, either in the R or SAS space, to set this?

I assume I can't configure SAS to run the R job on another server, and still allow users to use PROC IML to directly submit R jobs (and take advantage of the quick data passing between the two).

Re: Limit memory available to R (through PROC IML)

SASKiwi — Mon, 02 Aug 2021 20:33:07 GMT

Surely you should look at R's own memory settings? SAS has no control over what memory other software uses.

Re: Limit memory available to R (through PROC IML)

Kurt_Bremser — Mon, 02 Aug 2021 20:40:09 GMT

Limit the memory consumption of user processes by means of the operating system.
A well set-up system should never allow a user process to bring it down.

Re: Limit memory available to R (through PROC IML)

snoopy369 — Mon, 02 Aug 2021 20:43:07 GMT

It could be set in either place (SAS could set the limit when it spawns the process), but either way I hope to get an answer from someone who administers a SAS server as to how they set that limit whether or not it’s in SAS configuration.

Re: Limit memory available to R (through PROC IML)

snoopy369 — Mon, 02 Aug 2021 20:43:51 GMT

Windows actually was okay, but it killed the SAS WebAppServer as that was the second biggest consumer of memory.

Re: Limit memory available to R (through PROC IML)

gwootton — Mon, 02 Aug 2021 20:59:41 GMT

Based on this documentation it sounds like SAS doesn't pass options to R when it starts it but you can set environment variables:
No startup options are sent to R by the SUBMIT statement. In particular, you cannot set command-line options such as --no-restore or --vanilla. However, you can set some command-line options by using environment variables.
Source: https://go.documentation.sas.com/doc/en/pgmsascdc/9.4_3.5/imlug/imlug_r_sect004.htm?

So perhaps you can use R_MAX_MEM_SIZE to limit memory consumption from R?

https://www.rdocumentation.org/packages/utils/versions/3.6.2/topics/memory.size

Re: Limit memory available to R (through PROC IML)

snoopy369 — Mon, 02 Aug 2021 21:40:15 GMT

Ah, that's just what I was hoping for - I looked at some documentation pages that talked about everything else in that _except_ the environment variable, for some reason... will try it out, thanks!