BookmarkSubscribeRSS Feed
UVakkalagadda
Obsidian | Level 7

Hello Chaps,

 

Seems to be a silly question ..  🙂 but I am looking for the best possible answer. I would like to know about the SAS permanent libraries limit in any environment. In my project we have around 2000+ libraries are created. I have one concern if we have that much of libraries will it impact to metadata performance? Appreciate your thoughts on this..~!

 

Thanks

Uday Vakkalagadda

Uday
11 REPLIES 11
JuanS_OCS
Amethyst | Level 16

Hello @UVakkalagadda,

 

I am not aware of any logical limit. You could share this with SAS Technical Support, to double check about limits. However:

 

I would consider the number of libraries pre-assigned by metadata, or assigned by autoexec: the more that are pre-assigned in a session, the longer it might take to start a SAS session and higher memory consumption. Normally, there is no much difference between, say, 10-20 libraries, but we are speaking here of 2000+, hence the consideration about time and memory of the SAS session.

 

Something else, not relevant to my remark shown above, is that perhaps you could base libraries definitions based on macro-variables. A technique a bit tricky sometimes, but if the problem is limits on metadata registration (which, I think, does not exist) you could reduce on such way.

UVakkalagadda
Obsidian | Level 7

Hi Juan,

 

The libraries not pre-assigned and its mixed of base, hadoop and LASR libraries.

Uday
SASKiwi
PROC Star

Are you pre-assigning the libraries or not? Each time you open a SAS session I would expect a significant overhead to assign 2,000 libraries. Also are these SAS libraries only, or are there a lot of libraries pointing to external databases? Setting up connections to external databases is likely to use even more resources.

 

I would suggest using the DEFER = YES option on any external database connections. You may also want to consider having some SAS libraries pointing to multiple folders to reduce the number of libraries required.

 

I'm not aware of any fixed limit on the number of libraries you can set up but the more you have the higher the overhead you will have on SAS session startup. Only trialing the use of so many will prove if performance is acceptable or not.

UVakkalagadda
Obsidian | Level 7

All are not pre-assigned but majority libraries are from Hadoop and LASR related libraries. I think those are created when user select PUBLIC LASR load option. Right now we don't have that much usage on the platform but in future it may impact. So, I am trying to  figure it out before hit the performance.

Uday
SimonDawson
SAS Employee

What are the libraries used for? There might be another way to tackle this?

Is this a library per user or something like that?

UVakkalagadda
Obsidian | Level 7

We have different libraries majority are from Hadoop and LASR. I could see each user is having 3 different types of duplicate libraries for LASR.

Uday
ballardw
Super User

The large number of libraries makes me think that you may want to investigate assigning single libraries to aggregate locations. A single library can reference multiple directories.  As long as the datasets within the folders don't duplicate names there shouldn't be any problem.

 

For example from my install the SASHELP library is related to 7 directories (and some of

 

Library Name Pathname
SASHELP C:\Program Files\SASHome2\SASFoundation\9.4\aacomp\sashelp
SASHELP C:\Program Files\SASHome2\SASFoundation\9.4\cmp\sashelp
SASHELP C:\Program Files\SASHome2\SASFoundation\9.4\core\sashelp
SASHELP C:\Program Files\SASHome2\SASFoundation\9.4\graph\sashelp
SASHELP C:\Program Files\SASHome2\SASFoundation\9.4\nls\en\SASCFG
SASHELP C:\Program Files\SASHome2\SASFoundation\9.4\spdsclient\sashelp
SASHELP C:\Program Files\SASHome2\SASFoundation\9.4\stat\sashelp
UVakkalagadda
Obsidian | Level 7

Right now no impact but when ever we connect to SAS EG LASR App we are getting the following warning message

" Multiple libraries were detected in metadata use the same libref for server SASApp. This is not a supported configuration and ma cause problems. Please contact your SAS administrator"

Uday
SASKiwi
PROC Star

Why do you have multiple libraries pointing at the same LASR servers?

UVakkalagadda
Obsidian | Level 7

Not sure about that we are investigating..! But most of the libraries are not pre-assigned.

Uday

suga badge.PNGThe SAS Users Group for Administrators (SUGA) is open to all SAS administrators and architects who install, update, manage or maintain a SAS deployment. 

Join SUGA 

CLI in SAS Viya

Learn how to install the SAS Viya CLI and a few commands you may find useful in this video by SAS’ Darrell Barton.

Find more tutorials on the SAS Users YouTube channel.

Discussion stats
  • 11 replies
  • 1726 views
  • 6 likes
  • 6 in conversation