Help organizing PATH-caslibs with complex folder structures

linjan — Mon, 02 Sep 2024 15:57:43 GMT

Hello! I need some help/tips on structuring/organizing PATH-caslibs for a "semi-multitenant" SAS Viya Kubernetes installation.

I only have one cas-server (cas-shared-default) that is used by two separate groups of users from org1 and org2.

Org1 and org2 each have a folder on an NFS-mount, and they're creating their own subfolders structures containing datasets (xlsx and csv files).

The folder structures are as shown below.

/nfs-mount/
├── org1/
│   ├── 202405_ListingA.xlsx
│   ├── 202405_ListingB.xlsx
│   ├── 202407_ListingA.xlsx
│   ├── 202407_Listingb.xlsx
│   ├── imports/
│   │   ├── 202406_areas.xlsx
│   │   ├── 202404_codes.xlsx
│   │   ├── 202405_definitions.xlsx
│   │   └── 202404_names.xlsx
│   ├── tests.csv
│   └── code.sas
├── org2/
│   ├── 'Codes and groups.xlsx'
│   ├── 'Codes and groups (2024-07-01).xlsx'
│   ├── 2024-07-01/
│   │   ├── f-codes.xlsx
│   │   ├── f-totals.xlsx
│   │   ├── i-codes.xlsx
│   │   └── i-totals.xlsx
│   ├── 2024-09-01/
│   │   ├── f-codes.xlsx
│   │   ├── f-totals.xlsx
│   │   ├── i-codes.xlsx
│   │   └── i-totals.xlsx
│   └── history/
│       ├── f/
│       │   ├── f-totals001.xlsx
│       │   ├── f-totals002.xlsx
│       │   └── f-totals003.xlsx
│       └── i/
│           ├── i-totals001.xlsx
│           ├── i-totals002.xlsx
│           └── i-totals003.xlsx
└── common/
    ├── exports.csv
    ├── definitions.csv
    └── distributions/
        ├── distribution1/
        │   └── exports.csv
        └── distribution2/
            └── exports.csv

So their respective folders are structured differently. And they also share a folder called "common" that has yet another folder structure.

I first created a global PATH-caslib for each top-folder. I.e.

caslib ORG1 with path = /nfs-mount/org1
caslib ORG2 with path = /nfs-mount/org2
caslib COMMON with path = /nfs-mount/common

However, in SASDataExplorer those PATH-caslibs only show files from the top-folder of the path as tables. Files in subfolders are ignored and can't be loaded into memory from there. And I can't create another PATH-caslib for the subfolders as I get the following error.

ERROR: The caslib ORG1_IMPORTS is a duplicate, parent or subpath of caslib ORG1.

How do you normally work with global PATH-caslibs to structured folders? Users from org1 and org2 don't want to put their data files in their respective root folder; they want their source data files structured. And they don't want to wait for a SAS Administrator to update global caslibs whenever a folder is created/edited/deleted.

Logically, the folder org1 is considered as its own "database", but it doesn't seem possible to map all its subfolders to a single global caslib (that is then mapped to a single library). It doesn't seem like I can connect multiple caslibs into a single library either.

The only (disliked) solution I can see is to create caslibs per subfolder (and keep these as updated on request). And disallow sub-subfolders. I was kind of hoping that at least the Import function in SASDataExplorer could load subfolders, but I can only get it load files of a folder into a single table (that won't work with org1's and org2's data).

Perhaps there's some magic SAS code that can load subfolder data into separate in-memory tables? That converts the relative path to a unique table name.

Any insights or descriptions of how others handle this scenario are welcome.

Re: Help organizing PATH-caslibs with complex folder structures

Majkee — Fri, 20 Sep 2024 15:27:47 GMT

I also need a solution for this. Anyone?

topic Re: Help organizing PATH-caslibs with complex folder structures in Administration and Deployment

Help organizing PATH-caslibs with complex folder structures

Re: Help organizing PATH-caslibs with complex folder structures