SAS Data Integration Studio, DataFlux Data Management Studio, SAS/ACCESS, SAS Data Loader for Hadoop and others

Getting a List of Metadata Folders

Reply
Regular Contributor
Posts: 155

Getting a List of Metadata Folders

Is there a defined prompt for the list of meta folders that I can use in a DI Studio user transform? I would like something like the file prompt, but I don't see that.

If there is no such prompt, what code could I run in DI Studio to create a data set containing that information?
Super User
Posts: 10,020

Re: Getting a List of Metadata Folders

Hi.
Maybe Dictionary table dictionary.libnames will contains information you need.



Ksharp
Regular Contributor
Posts: 155

Re: Getting a List of Metadata Folders

It does not. There is a metadata prompt object for the libraries defined in the metadata which returns both the libnames and metadata folder paths. What I need to do is prompt the user with a list of all the metadata folders so the user can select which folder to register metadata in.
SAS Employee
Posts: 51

Re: Getting a List of Metadata Folders

Hi Don,

Can you tell me a little more about this use case? The only reason I can think of to prompt the user with a list of metadata folder locations is if your transformation is creating metadata objects and you need the user to specify a folder where they should be located. Is this the case? If so, please describe as there may be other options.

Thanks,

Tim Stearn
Regular Contributor
Posts: 155

Re: Getting a List of Metadata Folders

Posted in reply to TimStearn_SASProductManagement_
Sure. I just took the DI Fast Track Course and I decided I wanted/needed to write a transformation to dynamically split data based on conditions in the data (e.g., values of a variable). For performance reasons the looper is not a good option as that passes the entire input table for each value.

I've already written the code to do the splitting in a single data step and have packaged that as a transformation that I call the Dynamic Splitter (and BTW, the instructors really liked it and were interested in including it in the course as an example).

I also added a Y/N prompt for whether all the output tables should be registered.

I use the libname prompt object to allow the user to specify where to put the output data sets. That gives me the information I need both for both the data step (the libname) and proc metadata (for the library specification). What I am missing is the ability to specify the folder location for where the metadata should be saved. I know I could have the user type it in, but I would prefer not having to do that.
SAS Employee
Posts: 51

Re: Getting a List of Metadata Folders

Don,

I see. Unfortunately, there is no prompt type defined for this type of requirement. While there are programmatic ways of retrieving a folder list, I'm guessing you're looking for a control that will allow the user to walk through the folders like the "Save As" dialog. I'm afraid you will need to resort to having the user type the location. You can programmatically validate whether this is a valid folder location - please let me know if you need any tips here.

I can add this to our list of requirements for consideration in a future version. In terms of the code inefficiency that led you down this path in the first place, SAS is looking to address this in the version following DI Studio 4.3 for this specific problem and in general.

Thanks,

Tim Stearn
Regular Contributor
Posts: 155

Re: Getting a List of Metadata Folders

Posted in reply to TimStearn_SASProductManagement_
Thx Tim for adding it to the list.

I would be interested in code to validate an entered entry.

And If you can provide code that can be run by DI Studio to query the metadata and return the list of folders, I can make that work as well using a dynamic data list selector.
SAS Employee
Posts: 51

Re: Getting a List of Metadata Folders

Don,



I can send the code, but I'm afraid things could get pretty convoluted if you try to do this dynamically for user presentation. It could be done (though not terribly well) through dynamic prompts that query database tables:

  1. Write a program that queries the metadata folder hierarchy and then writes the folder entries to a table or set of tables.
  2. Run this program at the desired frequency to keep the tables (and folder list) up to date.
  3. Use the capability of prompts to query a database table to present the folder list. Here's where things get a bit challenging, since you have to choose an effective way to present the hierarchy to a user without the capability to let them drill down incrementally. They could be faced with a very long list with a indeterminant number of indented levels.

Maybe I'm not thinking of a more creative way that one could approach this, so if you've got one, go ahead and give it a shotSmiley Wink.



I'll send some DATA Step code tomorrow night that shows you how to query for a folder and a folder's children. You should be able to use this code to implement validation and folder display.




Thanks,



Tim Stearn
Regular Contributor
Posts: 155

Re: Getting a List of Metadata Folders

Posted in reply to TimStearn_SASProductManagement_
Thx Tim.

I have a couple of ideas that might make this a bit less kludgier. One thought - if this could be packaged inside a data step using the query functions instead of a proc metadata step, I wonder if defining the data step as a view might help????

Regardless, once you send me some code, I will tinker with it and so if I can come up with something.

Thx again!
SAS Employee
Posts: 51

Re: Getting a List of Metadata Folders

Don,

Sorry it took me a while to get this too you - I just got a chance to sit down and hack it out tonight. And....it turned out to be more involved than I thought it would be. Most of the code is related to determining that the path passed in is valid.

I used PROC FCMP functions to make this work since I needed a recursive approach to walk the folder path. I've included a simple DATA Step program to show you how to use the functions.

You'll notice serveral "meta" variables before the call (metaserver, metaport, etc). If you call this function within DI Studio, you won't need to set these as they'll already be set.

Please take a look and let me know if you have questions.

Thanks,

Tim Stearn
Regular Contributor
Posts: 155

Re: Getting a List of Metadata Folders

Posted in reply to TimStearn_SASProductManagement_
Thx. And no need to apologize. I am swamped as well. Will try to check this out over the weekend.

But I do have to add that it seems to me that something as basic as getting the list of folders should be simple to do? I would think that this is pretty common. So the fact that this was not easy perhaps strengthens the argument that it should be implemented as a supported prompt.
SAS Employee
Posts: 51

Re: Getting a List of Metadata Folders

Don,

I just realized the code I passed along to you isn't 9.2 compliant. There is an issue with PROC FCMP in 9.2 that won't allow this code to run. I just realized I had built/tested this on pre-release 9.3 software. Please give it a look and see if you're comfortable adapting it. If not, I can spend some time on it next week.

Thanks,

Tim Stearn
New Contributor
Posts: 4

Re: Getting a List of Metadata Folders

Posted in reply to TimStearn_SASProductManagement_

Hi Tim,

I am trying to get the list of folders, subfolders and the files in it. Don't have Dis. is it possible for me to get those information form SMC. If so, How can i do that?

Thanks

Contributor
Posts: 66

Re: Getting a List of Metadata Folders

One way you can do it albeit indirectly is via the SAS Security macros.  It will give out a list of folders and the permissions of each group,  It's an autocall so just run %mdsecds; You will get a bunch of datasets in work.  The view mdsecds_join will show the folders.

In your prompt populate the prompt from data in these datasets.  you will also get the metadata id.  You will then need to write some logic to issue an addMetadata/UpdateMetadata call as well to get the table information into the repository.

New Contributor
Posts: 4

Re: Getting a List of Metadata Folders

Posted in reply to CameronLawson

Hi,

I am going to try this. Will let you know the result.

Thanks

Ask a Question
Discussion stats
  • 14 replies
  • 1662 views
  • 0 likes
  • 5 in conversation