DATA Step, Macro, Functions and more

SAS 9.4 Extract long texts from SAS Metadata TextStore

Accepted Solution Solved
Reply
Contributor
Posts: 34
Accepted Solution

SAS 9.4 Extract long texts from SAS Metadata TextStore

Hi everybody

We run a traditional data warehouse with several thousand jobs built in DI Studio and a lot of stored processes. There is a lot of user written code stored in SAS metadata, and we want to extract the SAS source code from SAS metadata TextStore and store it in disk files, which makes it much easier to scan for used macros, file names etc. There are about 2000 entries containing user written code from DI Studio transformations, pre- and post-process code and code used in Stored processes.

 

I use an automated process flow that extracts metadata to XML, reads the xml file into a SAS data sets using a XML MAP and - in a flllowing data step - formats the StoredText content to SAS programs and writes each entry to a disk file with the TextStore ID as file name.  The link between files and jobs/transformations are maintained in tables, which are written along with the extraction.

 

It works very well. But some entries are longer than 32k, and they are truncated when read into the SAS data set. There seems to be no workaround like using the XML map to split the long texts in several records, so I have not figured out how to get content beyond the 32k limit out in disk files. 

 

I hope someone has a good idea to solve this problem.

 

 

 

 


Accepted Solutions
Solution
‎01-23-2017 11:56 AM
Super User
Posts: 5,427

Re: SAS 9.4 Extract long texts from SAS Metadata TextStore

Posted in reply to ErikLund_Jensen
Step one: fire the ETL designer/development lead for allowing that amount of user written code. As you experience now, it's a damage that is costly to repair.

For the XML, since the XML mapper nor Base SAS handles this, you probably need to use a 3rd party tool to do this for you.

I haven't dug i to Metadata behind the scenes that much, but it's stored in Base SAS tables, so I guess that the original store would be splittede into multiple rows...or. ..?
Data never sleeps

View solution in original post


All Replies
PROC Star
Posts: 1,760

Re: SAS 9.4 Extract long texts from SAS Metadata TextStore

Posted in reply to ErikLund_Jensen

Sadly I can't offer help, but SAS should seriously work on lifting the limits for character variable length and metadata length.

The increase from 200 to 32k is 20 years old.

Solution
‎01-23-2017 11:56 AM
Super User
Posts: 5,427

Re: SAS 9.4 Extract long texts from SAS Metadata TextStore

Posted in reply to ErikLund_Jensen
Step one: fire the ETL designer/development lead for allowing that amount of user written code. As you experience now, it's a damage that is costly to repair.

For the XML, since the XML mapper nor Base SAS handles this, you probably need to use a 3rd party tool to do this for you.

I haven't dug i to Metadata behind the scenes that much, but it's stored in Base SAS tables, so I guess that the original store would be splittede into multiple rows...or. ..?
Data never sleeps
Contributor
Posts: 34

Re: SAS 9.4 Extract long texts from SAS Metadata TextStore

Thanks to LinusH !

 

Stupid I did'nt think of that myself. Of course the texts can be extracted from Metadata tables, and for my purpose a libname to the most recent backup is sufficient. I ran into a problem with transcoding danish national characters from Unicode, but that can be solved.

 

☑ This topic is solved.

Need further help from the community? Please ask a new question.

Discussion stats
  • 3 replies
  • 180 views
  • 0 likes
  • 3 in conversation