BookmarkSubscribeRSS Feed
🔒 This topic is solved and locked. Need further help from the community? Please sign in and ask a new question.
ErikLund_Jensen
Rhodochrosite | Level 12

Hi everybody

We run a traditional data warehouse with several thousand jobs built in DI Studio and a lot of stored processes. There is a lot of user written code stored in SAS metadata, and we want to extract the SAS source code from SAS metadata TextStore and store it in disk files, which makes it much easier to scan for used macros, file names etc. There are about 2000 entries containing user written code from DI Studio transformations, pre- and post-process code and code used in Stored processes.

 

I use an automated process flow that extracts metadata to XML, reads the xml file into a SAS data sets using a XML MAP and - in a flllowing data step - formats the StoredText content to SAS programs and writes each entry to a disk file with the TextStore ID as file name.  The link between files and jobs/transformations are maintained in tables, which are written along with the extraction.

 

It works very well. But some entries are longer than 32k, and they are truncated when read into the SAS data set. There seems to be no workaround like using the XML map to split the long texts in several records, so I have not figured out how to get content beyond the 32k limit out in disk files. 

 

I hope someone has a good idea to solve this problem.

 

 

 

 

1 ACCEPTED SOLUTION

Accepted Solutions
LinusH
Tourmaline | Level 20
Step one: fire the ETL designer/development lead for allowing that amount of user written code. As you experience now, it's a damage that is costly to repair.

For the XML, since the XML mapper nor Base SAS handles this, you probably need to use a 3rd party tool to do this for you.

I haven't dug i to Metadata behind the scenes that much, but it's stored in Base SAS tables, so I guess that the original store would be splittede into multiple rows...or. ..?
Data never sleeps

View solution in original post

3 REPLIES 3
ChrisNZ
Tourmaline | Level 20

Sadly I can't offer help, but SAS should seriously work on lifting the limits for character variable length and metadata length.

The increase from 200 to 32k is 20 years old.

LinusH
Tourmaline | Level 20
Step one: fire the ETL designer/development lead for allowing that amount of user written code. As you experience now, it's a damage that is costly to repair.

For the XML, since the XML mapper nor Base SAS handles this, you probably need to use a 3rd party tool to do this for you.

I haven't dug i to Metadata behind the scenes that much, but it's stored in Base SAS tables, so I guess that the original store would be splittede into multiple rows...or. ..?
Data never sleeps
ErikLund_Jensen
Rhodochrosite | Level 12

Thanks to LinusH !

 

Stupid I did'nt think of that myself. Of course the texts can be extracted from Metadata tables, and for my purpose a libname to the most recent backup is sufficient. I ran into a problem with transcoding danish national characters from Unicode, but that can be solved.

 

sas-innovate-2024.png

Join us for SAS Innovate April 16-19 at the Aria in Las Vegas. Bring the team and save big with our group pricing for a limited time only.

Pre-conference courses and tutorials are filling up fast and are always a sellout. Register today to reserve your seat.

 

Register now!

How to Concatenate Values

Learn how use the CAT functions in SAS to join values from multiple variables into a single value.

Find more tutorials on the SAS Users YouTube channel.

Click image to register for webinarClick image to register for webinar

Classroom Training Available!

Select SAS Training centers are offering in-person courses. View upcoming courses for:

View all other training opportunities.

Discussion stats
  • 3 replies
  • 906 views
  • 0 likes
  • 3 in conversation