How many permanent libraries SAS environment can hold? Is there any limit?

[UVakkalagadda]

Hello Chaps,

Seems to be a silly question ..  🙂 but I am looking for the best possible answer. I would like to know about the SAS permanent libraries limit in any environment. In my project we have around 2000+ libraries are created. I have one concern if we have that much of libraries will it impact to metadata performance? Appreciate your thoughts on this..~!

Thanks

Uday Vakkalagadda

[JuanS_OCS]

Hello @UVakkalagadda,

I am not aware of any logical limit. You could share this with SAS Technical Support, to double check about limits. However:

I would consider the number of libraries pre-assigned by metadata, or assigned by autoexec: the more that are pre-assigned in a session, the longer it might take to start a SAS session and higher memory consumption. Normally, there is no much difference between, say, 10-20 libraries, but we are speaking here of 2000+, hence the consideration about time and memory of the SAS session.

Something else, not relevant to my remark shown above, is that perhaps you could base libraries definitions based on macro-variables. A technique a bit tricky sometimes, but if the problem is limits on metadata registration (which, I think, does not exist) you could reduce on such way.

[UVakkalagadda]

Hi Juan,

The libraries not pre-assigned and its mixed of base, hadoop and LASR libraries.

[Kurt_Bremser]

That many libraries look supicious to me. How are users supposed to keep track of them?

We keep ~100000 datasets in less than 100 libraries.

[SASKiwi]

Are you pre-assigning the libraries or not? Each time you open a SAS session I would expect a significant overhead to assign 2,000 libraries. Also are these SAS libraries only, or are there a lot of libraries pointing to external databases? Setting up connections to external databases is likely to use even more resources.

I would suggest using the DEFER = YES option on any external database connections. You may also want to consider having some SAS libraries pointing to multiple folders to reduce the number of libraries required.

I'm not aware of any fixed limit on the number of libraries you can set up but the more you have the higher the overhead you will have on SAS session startup. Only trialing the use of so many will prove if performance is acceptable or not.

[UVakkalagadda]

All are not pre-assigned but majority libraries are from Hadoop and LASR related libraries. I think those are created when user select PUBLIC LASR load option. Right now we don't have that much usage on the platform but in future it may impact. So, I am trying to  figure it out before hit the performance.

[SimonDawson]

What are the libraries used for? There might be another way to tackle this?

Is this a library per user or something like that?

[UVakkalagadda]

We have different libraries majority are from Hadoop and LASR. I could see each user is having 3 different types of duplicate libraries for LASR.

[ballardw]

The large number of libraries makes me think that you may want to investigate assigning single libraries to aggregate locations. A single library can reference multiple directories.  As long as the datasets within the folders don't duplicate names there shouldn't be any problem.

For example from my install the SASHELP library is related to 7 directories (and some of

Library Name	Pathname
SASHELP	C:\Program Files\SASHome2\SASFoundation\9.4\aacomp\sashelp
SASHELP	C:\Program Files\SASHome2\SASFoundation\9.4\cmp\sashelp
SASHELP	C:\Program Files\SASHome2\SASFoundation\9.4\core\sashelp
SASHELP	C:\Program Files\SASHome2\SASFoundation\9.4\graph\sashelp
SASHELP	C:\Program Files\SASHome2\SASFoundation\9.4\nls\en\SASCFG
SASHELP	C:\Program Files\SASHome2\SASFoundation\9.4\spdsclient\sashelp
SASHELP	C:\Program Files\SASHome2\SASFoundation\9.4\stat\sashelp

[UVakkalagadda]

Right now no impact but when ever we connect to SAS EG LASR App we are getting the following warning message

" Multiple libraries were detected in metadata use the same libref for server SASApp. This is not a supported configuration and ma cause problems. Please contact your SAS administrator"

[SASKiwi]

Why do you have multiple libraries pointing at the same LASR servers?

[UVakkalagadda]

Not sure about that we are investigating..! But most of the libraries are not pre-assigned.