Moving SAS Model Studio Custom Pipelines to a New Environment
Moving Model Studio custom pipelines from one environment to another can be easy and useful. But be sure to read the fine print! Some of the things that you thought might convey don’t actually come along.
Perhaps you’ve bought or sold a house recently. Some things convey with the house, such as the shutters, storm window, and the irises out front.
But maybe it turns out that some things that you thought would convey, don’t convey after all!
And suddenly you find yourself buying a new washer and dryer and a bunch of blinds.
The lesson here is: Read the small print! Make sure you are aware what will come along with your house.
Likewise when migrating or moving things around in SAS Viya, it's important to be aware of what comes along and what has to be migrated separately or rebuilt upon arrival.
In this article I will cover just one aspect of this...moving custom pipelines.
Creating Custom Pipeline in Model Studio (SAS Viya 3.5)
In Model Studio, let’s create a New Project from the Intermediate with Class Target template. Notice that none of the variables is automatically assigned as the Target.
Let’s first look at our global metadata. We see no global metadata to display.
Mobile users: To view the images, select the "Full" version at the bottom of the page.
We’ll select BAD and assign it as the target.
We’ll then change DELINQ from Level: Nominal to Interval and Impute a Custom constant value of 0.
Next we select both BAD and DELINQ and use the icon to assign these to global metadata.
If we open our Global Metadata pane, we see our two variables BAD and DELINQ, with our metadata adjustments.
Now let’s move to the pipeline tab and will add a manage variables node following the data node, use the Ctrl key to select both Imputation and Decision Tree nodes, right clicking and selecting Add parent node > Data Mining Preprocessing > Manage Variables.
We’ll right-click and run the Manage Variables node, and then select Manage Variables.
Select CLNO and change Role to Rejected.
Close and Save to Manager Variables node.
We do NOT save this to Global Metadata. Close. Run the pipeline, name it MyCustomPipe and then Save it to the Exchange.
We’ll keep the default name (which adds _1 to the pipeline name) and Save.
Now let’s sign out and come in as a different user. Create a New Project and we’ll call it Tester300. We’ll browse the templates and find our custom template.
Then we will browse for the same data set.
Looking first at the Data tab we see that BAD has come in as the Target and that DELINQ is interval with a Custom constant value of 0.
So our global metadata persisted within the same environment!
Let’s look at our pipeline and run the manager variables node and look at it. We see that CLNO has the role of INPUT, so our designation of REJECTED did not persist when we opened this custom pipeline even in the same environment.
Exporting Pipeline
Let’s go to our Templates pane and download our original custom pipeline.
It downloads as a zip file.
Importing Pipeline in the new SAS Viya
Let’s see what happens when we try to open our custom template in a new SAS Viya environment. First we’ll upload the zip file to The Exchange (as a user without admin privileges).
Then we’ll start a New Project with the BethCustomPipe_1 template that we just uploaded. And we’ll open the project. Notice that BAD no longer comes in as the Target and DELINQ no longer comes in as interval. None of the global metadata is included with the zip file for the pipeline.
You might think that this is because we switched from a Viya 3.5 to a Viya 4.0 environment. No. Global metadata cannot be exported and imported among different environments even if they are the same version.
Let’s look at our pipeline. The pipeline structure has been preserved from Viya 3.5 to Viya 4.0.
However, we now know that any changes we made to data metadata must be repeated again in the new environment.
Some Tidbits on Migration
Starting with 2021.1.1 Stable, you no longer need to be administrator to import and export content.
Importing Model Studio content
- Users must exist and must have logged on in the target environment (or ownership must be modified – see import-converter tool)
- Data must be available in the target environment. Data must loaded to memory (in the same caslib as in the source environment) before project import
After importing:
- All nodes and pipelines must be rerun after migration to regenerate the results
- Models must be republished to CAS/MAS
As always, remember:
- Some metadata types/roles may be automatically detected, while others may not. For example, the software may automatically detect a binary variable. But it will never detect a target (dependent) variable, even if the variable name is “target.”
- Variables with numeric values, including binary, unary, and things like social security numbers and zip codes, are auto classified as Measures.
- The default aggregation method is sum, although in many analytics processes, we will want to use average. You can easily change this, but it is another manual step.
- Partitioning is not automatic.
For More Information
