topic how large is work library? in SAS Programming

how large is work library?

wang267 — Tue, 14 Aug 2018 03:00:30 GMT

Hi everyone,

I am just curious about how large is the Work library. Will it be able to contain a data set with 1 billion observations and 100 variables?

However, in IML, a 10000 * 10000 matrix cannot be simulated at one time. Log said something like "need more bytes". Is this related to size of work library or computer memory? Or, work library size is related to computer memory ?

Re: how large is work library?

ChrisNZ — Tue, 14 Aug 2018 05:05:24 GMT

1. The WORK library is typically a folder on your disk.

2. Please give the exact log messages.

3. I strongly suspect that your IML message is referring to insufficient virtual memory, i.e. RAM + paging space.

This space is managed by option MEMSIZE. Look up the documentation regarding this option.

4. Be aware that increasing MEMSIZE beyond available RAM might allow your process to complete, but very slowly. Paging (i.e. using disk as if it were RAM) is a trade-off: speed is sacrificed to enable process completion.

Re: how large is work library?

Patrick — Tue, 14 Aug 2018 12:24:19 GMT

@wang267

SAS Work is disk space and it can become as big as there is space on the disk (or the quota you've got).

Always post the exact log message. The one you reference is likely something similar to below:

https://communities.sas.com/t5/SAS-IML-Software-and-Matrix/Unable-to-allocate-sufficient-memory/td-p/214672

Re: how large is work library?

Reeza — Tue, 14 Aug 2018 14:44:36 GMT

@wang267 wrote:

Hi everyone,

I am just curious about how large is the Work library. Will it be able to contain a data set with 1 billion observations and 100 variables?

However, in IML, a 10000 * 10000 matrix cannot be simulated at one time. Log said something like "need more bytes". Is this related to size of work library or computer memory? Or, work library size is related to computer memory ?

It depends. At my old work place we each had a 2GB limit on the work library and had to get it removed if we worked with big data. At other places there were no restrictions beyond my computer size. You can also redirect work to a network drive which would likely have unlimited space but your read/write time will increase slowing everything down.