Export and import metadata for all stored processes

[AMK]

Hi,

we are moving from SAS 9.1 to SAS 9.4, on new hardware and I have been manually recreating stored processes in the new environment based on the code etc. from the old environment, however it is very slow and boring and I'm thinking there has to be an easier way!

Is it possible to export the metadata for all the stored processes in our old environment in such a way that I can then tweak the metadata so that it will work in our new environment and then import the metadata into our new environment?

I saw on the sas support site that there is some java tool that can do it, but there's no mention of a name for it or where to find it, so I am stumped.

Thanks,

Adrian.

[sandykota]

Hi Adrian,

You can export the required stored process as a package (.spk) from your old environment and copy into some media. And then import the package (.spk) into new environment.

Hope the below scenario will help you:

http://support.sas.com/documentation/cdl/en/bisag/68240/HTML/default/viewer.htm#p1k1twndkw9je7n1sq3y...

[AMK]

Thanks sandykota, however I don't think the "Folder" structure mentioned in that documentation exists in the Management Console in SAS 9.1, so I can't make a package to export.

[VasilijNevlev]

Hello Amk, 

What sort of changes are you planning to make to the metadata? This is important, as not all changes are supported by the tools I am about to discribe. 

You might have SASObjectFramework installed on your PC or the server. Look for it in your SASHome folder. The rule of thumb, is you have SMC or SASDI installed, you will have SASPlatformObjectFramework too. It should be something like: "C:\Program Files\SASHome\SASPlatformObjectFramework\9.4" 

There are two tools that you need: 

* ExportPackage.exe - this tool has all the functionality of your SMC package tool. You need to provide to it a list of objects, the profile name, package name and the object type, the tool will do the rest. To see full list of commands and examples, just launch the exe file in CMD without any parameters, it will output the command line help for you. 

* ImportPackage.exe - this tool is the reverse of the above, it imports the package while taking into account the substitution file (text file describing what changes you want to be made to the mappings). Via a command line you need to specify the connection profile, package name and destination in SAS Folders. 

* sas-list-objects.exe - this tool allows to to export out the full list of all of SAS metadata objects based on a search criteria. Really useful to generate a list of objects to be inputed into ExportPackage.exe You will find this tool in "tools" in the root folder for your SASPlatformObjectFramework. 

This tools sometimes isn't that easy to use, post in here the challenges you will find and I will help you to overcome them. 

Regards,

Vasilij

[AMK]

Hi Vasilij,

thanks for your suggestions and offer of help, however I cannot see those executables in SAS 9.1 and from talking with Tech support in parallel, it looks like it's not possible to migrate directly from 9.1 to 9.4. Instead we would need to migrate to 9.2 first and from there to 9.4, but obviously we don't have an install of 9.2, so we cannot do that.

I have asked Tech Support if it's possible to take one of the stored processes I have set up in 9.4 and to use it (somehow) as a template. The only thing that changes on our stored processes is the name, URL and .sas file, so if I can figure out what needs to be changed in the template, I should be able to script the migration of the last 200 stored processes.

Thanks,

Adrian.

[VasilijNevlev]

Hello Amk,

Are you able to manually export all 200 STPs as one SPK? Or that where the problem is?
If you export STP from your environment, what happens when you import that package into 9.4? It should offer you a mapping screen where you can udpate the parameters that you want.

Let me know. We can figure this out.

Regards,
Vasilij

[JuanS_OCS]

Hello @AMK,

if you are moving from 9.1 to 9.4, the best option you have is to migrate your environment in 2 stages, from 9.1 to 9.3 and from 9.4 to 9.4, instead of manual promotion. As you said, the re are no proper tools otherwise. As you said as well, metadata structure changed, tools changed...

Please give a look to thos Paper, it explains the process: http://support.sas.com/resources/papers/proceedings15/SAS1801-2015.pdf

About the tool you mention, is the SMU tool, the SAS migration utility. it comes with every SAS depot now, but in the SAS 9,4 depot there is no tool to migrate from SAS 9.1, that is why the migration in 2 steps.

http://support.sas.com/rnd/migration/utility/index.html

http://support.sas.com/documentation/cdl/en/bisag/68240/HTML/default/viewer.htm#p04l8c3fo8tpyhn103dp...

Does it help?

Kind regards,

Juan

[AMK · Accepted solution]

Thanks for all your suggestions, but in the end I resorted to a combination of Python and SAS to do what I needed.

Tech support mentioned the MacroCore library on Github might be of some help, although it was at my own risk.

So first of all I used Python to query the stored process server for a list of all stored processes using &_debug=list, which also lists the location of the .sas files for each stored proc.

I then set up the metadata folders in Management Console to match the folder structure in the old environment and repeatedly used the macrocore %mm_gettree function to get the metadata object ids for each metadata folder.

I created a python script to copy every .sas file in the stored process folders on the old server over to the new server and a script to replace references to the old server with the new one, the old port with the new one etc.

After that I used a python script to generate sas code to run the %mm_CreateSTP function on all the stored processes, passing in the relevant data (stored process name, description, physical file location, metadata folder object id, tree object id, package/streaming flags) and ran this from SAS.

Finally I created another python script to run through all the files on our web server, looking for references to the old stored process server and fixing them so they pointed at the new stp server, on the new port and with the updated usernames and passwords.

In total, I actually ended up converting around 400 stored processes this way and updated over 3000 URLs. It was a bit tricky to figure out the correct data to supply to the macrocore functions, but once I got around that, it worked very well.

Thanks again,

Adrian.

[boemskats]

See @AllanBowe I told you GitHub stars don't mean anything 🙂 

[AllanBowe]

Good to know it's being used, and recommended by SAS Support no less 

@AMK, when you say "It was a bit tricky to figure out the correct data to supply to the macrocore functions", how do you mean - is there anything we could do to improve the documentation?