Where to define your SAS libraries in SAS Viya? Part 2

[#1]

After first part, let’s continue our journey on identifying where to define SAS libraries.

So far, we have discussed:

• The global way
• The contextual way
• The personal way

The UI way

Libraries can also be defined using a dedicated User Interface in SAS Studio. This is what Gerry’s blog mentioned in the first part is about.

Select any image to see a larger version.
Mobile users: To view the images, select the "Full" version at the bottom of the page.

Multiple cases can then happen:

1. A user with no administrative privilege creates a library and doesn’t check “Assign and connect to data sources at startup”

This library is created in the Data Sources microservice. Thus, it is available to a SAS Administrator in SAS Environment Manager as a resource that could be attached to a context (see later).

The library will be available as a "Disconnected Library" at next user’s session, regardless of the context. It is tied to a user.

It is not added to the user’s SAS Studio autoexec.

2. A user with no administrative privilege creates a library and does check “Assign and connect to data sources at startup”

This library is created in the Data Sources microservice. Thus, it is available to a SAS Administrator in SAS Environment Manager as a resource that could be attached to any context (see later).

The library will be available as a “Connected Library” at next session for this user, regardless of the context, because it has been added to the user’s SAS Studio autoexec, through the LIBDEF= LIBNAME option.

3. A user with administrative privilege creates a library, checks “Allow all users to view the library connection” but doesn’t check “Assign and connect to data sources at startup”

This library is created in the Data Sources microservice. Thus, it is available to a SAS Administrator in SAS Environment Manager as a resource that could be attached to any context (see later).

This library is added to the current context (the one that the user is currently connected to) as a resource. So, it will be available to any user starting a session with this context, as disconnected or as connected (depending on the choice made in the radio box).

It is not added to the user’s SAS Studio autoexec, so it will not be available if the user switches to another context.

4. A user with administrative privilege creates a library, checks both “Assign and connect to data sources at startup” and “Allow all users to view the library connection”

This library is created in the Data Sources microservice. Thus, it is available to a SAS Administrator in SAS Environment Manager as a resource that could be attached to any context (see later).

This library is added to the current context (the one that the user is currently connected to) as a resource. So, it will be available to any user starting a session with this context, as disconnected or as connected (depending on the choice made in the radio box).

The library will be available as a “Connected Library” at next session for this user, regardless of the context, because it has been added to the user’s SAS Studio autoexec, through the LIBDEF= LIBNAME option.

This is where some confusion can occur because the library is attached to a context and is also defined in the user’s autoexec. So, if the user starts a session using the original context (the one that was used to define the library), the library is assigned twice (the first time from the context, the second time from the autoexec). In reality, it doesn’t really matter. The user sees only one library. But this helps understand how libraries are assigned.

 

Impacts of the UI way

Now that we know how to use the UI to define libraries, additional questions arise.

Where do I see that a library defined in the UI has been attached to a context?

Indeed, by checking “Allow all users to view the library connection”, an admin adds a library to the current context. But how does this materialize?

In SAS Environment Manager, you can observe the results of that action in the context:

The libraries associated with a context show up under Resources. The Assign check box depends on how the library has been set up (Add as disconnected / Add as connected). Thus, you can have libraries defined under Resources and libraries defined under Advanced in the autoexec.

How can I add data source definitions to other contexts?

Since data source definitions are available globally to a SAS admin, he can decide to attach them to any context. For example, if he wants to add the ADCONT1 and ADCONT2 libraries to another context (for example the SAS Job Execution compute context), he will edit the compute context, go into Resources, then click on the + sign:

Then, the SAS admin will select the definitions:

And customize them if needed (assign them by default, rename them):

Instead of providing all the connection details in a SAS code autoexec, can I use the library definitions from the UI (using the LIBDEF= LIBNAME option) at a global level (compute for example)?

Yes, I can. I just need to find the right data source definition URI. The quicker way is to check “Assign and connect to data sources at startup” when defining a data source in the UI and check the SAS Studio autoexec right after. You should see the exact syntax of what you would use:

Then you can edit the compute server (like shown in “the global way”, previous blog) and add the library using the LIBDEF= syntax:

Where can I find all data source libraries defined in the UI by SAS users and how can I obtain their URI?

An admin can list them when trying to add a library to a context (Select a resource dialog show above). But this won’t give the URI needed by the LIBDEF= LIBNAME option.

The other option is to use the SAS Viya REST API to list all data source definitions and eventually obtain their URI.

Here are some examples:

# Using curl
# List all
curl -k "https://${SAS_VIYA_END_POINT}/dataSources/providers/Compute/sourceDefinitions" \
   -H "Authorization: Bearer $VIYA_ACCESS_TOKEN"
# Query one
curl -k "https://${SAS_VIYA_END_POINT}/dataSources/providers/Compute/sourceDefinitions?filter=eq(name,'ADMIN_CONTEXT2')" \
   -H "Authorization: Bearer $VIYA_ACCESS_TOKEN"

# Using pyviyatools https://github.com/sassoftware/pyviyatools
callrestapi.py -m get -e "/dataSources/providers/Compute/sourceDefinitions?filter=eq(name,'ADMIN_CONTEXT2')"

Here is a sample output:

Recap

We have been able to see that we can define libraries at different levels and scopes:

• At the server level (compute, batch, connect, etc.)
• At the context level (for compute servers)
• At the user level for SAS Studio (which works across contexts)

We have also observed that libraries defined using the “New Library Connection” wizard in SAS Studio can be managed, attached to compute contexts as Resources and used in library assignment with the LIBDEF= LIBNAME option.

What’s next?

In the future, a central and universal approach will be used to create and manage data source connections for both SAS Cloud Analytic Services (CAS) and SAS servers (compute, batch, etc.). One unique connection will be used to define both a CASLIB (for CAS) and a library (for SAS servers). This will improve the user experience and greatly reduce the maintenance of data source definitions in SAS Viya.

This new “Connections” component already exists for CAS connections only (it is NOT yet available for SAS libraries in SAS Studio) and is available in the “Manage Data” menu (SAS Data Explorer):

Defining and maintaining a data source connection becomes easier, fully guided, with a quick access to a lot of options, quick access to the documentation and requirements directly from the connection dialog, ability to search for options and to see what changes have been made.

Thanks for reading.

Find more articles from SAS Global Enablement and Learning here.

[touwen_k]

@NicolasRobert Thank you for providing clarity on this topic. I have a question: we have defined domains for oracle databases schemas. When we add such oracle database library to the compute context, then users who are not in the domain are not able to start the compute session in SAS Studio at all. Is this how the context supposed to work? We thought it will load the compute session without the library. We have added this library to the autoexec of  the compute server. Then we have no problems and users can start compute session without this library if they have no permission for it. regards Karolina 

[thesasuser]

Hello
The library locations have been defined as "/delcontent/data/etc".
Wondering if this is a physical location on user accessible disk OR in the contents in the  web infrastructure data server?

[GuenterGreulich]

With the REST API I can list all data source definitions. Can I create the compute data source using the REST API? If yes, how looks the syntax?

[thesasuser]

Thanks @GuenterGreulich 
I am a newbie for SAS Viya. Can you please share the sample code, which I can experiment with.
Thanks

[NicolasRobert]

@touwen_k 

Hi Karolina and thanks for your message. I just tried and I did not reproduce the issue. I was able to start a Compute session even with a user who is not authorized to access the authentication domain. The Compute context started without the Oracle library as expected.

Which SAS Viya version are you using and what is your syntax?

Here is what I did:

[NicolasRobert]

@thesasuser 

Hello.

/gelcontent/data is a physical path accessible from the SAS Compute Server. It is not a path from SAS Content. 

[NicolasRobert]

@GuenterGreulich 

Hello.

I believe this is doable as per the documentation (https://developers.sas.com/rest-apis/dataSources-v3), especially this: https://developers.sas.com/rest-apis/dataSources-v3?operation=createSourceDefinition.

I don't have an example to share though.

[touwen_k]

@NicolasRobert  hello Nicolas, thank you for checking. We have just updated to version 2024.04. Our users have put the libraries in different tab Resources, and then you cannot start the compute session.

[GuidoG]

Dear @NicolasRobert 

In SAS9 it was possible to use meta engine to assign predefined libraries as they were describer in metadata.
for example: 

libname MYDATA meta library=“MYDATA“;

As libraries are now described in Data Sources microservice and meta engine is no longer available

- how can we now use "MYDATA" (i.e library name) to assign predefined library with statement in SAS code and avoid using the long URI with LIBRARYDEFINITION option.

and I mean SPRE, not CAS environment

If you are not the right person to ask, would you be so kind to pass the question to more suitable person
Thank you in advance!

[Sajid01]

Hello @GuidoG 
I think the "Personal Way" in Part 1 of this article gives you a hint.
One should be able to create a libref using libname statement referring to an accessible physical location.

[GuidoG]

Dear @Sajid01 
Thank you for consideration, although this might not seem to be the best idea, when you should work in environment with hundres of libraries and thousands of different projects.  Not mentioning the count of develpers or scheduled jobs.

So far we have used predescribed libaries also for the reason if we should relocate data for any reason. 
When we used LIBNAME META engine to assign only the libraries needed at that exact moment we prevented lag in session start caused by assigning all the libraries.  And when data was relocated (or authentication policy changed), changing the library description in metadata kept our jobs running without the hassel of changing the hardcoded library statements on all jobs.

Using autoexec for libary assignment does not address the challanges we are facing, thats why I am wondering if there is a simpler way available for us than using REST API to retrieve LIBRARY's URI searching by NAME and then creating dynamic LIBNAME statement with LIBRARYDEFINITION option