Creating an ASTORE in SAS VDMML Model Studio on SAS Viya

4 Likes

Once you have created your SAS Viya machine learning models, you may want to export them so that you can move them to another environment. For example, you may want to move them to a database such as Teradata, Hadoop or SAP Hana for in-database scoring. Or you may want to transfer models you’ve developed to a partner or customer who operates in a different environment. Or you may be migrating to a new environment. Regardless of the reason you have for moving your machine learning models, you can easily export them using Model Studio on SAS.

Most of SAS’s traditional statistical models (SAS Visual Statistics) are packaged for export in a single downloadable data step code file. On the other hand, most of SAS’s VDMML machine learning models are packaged for efficient transfer in two parts:

Part 1: Embedded processing score code, dmcas_epscorecode.sas

Part 2: An analytic store, _S0MEL0N6NUMBERLETTERC0MB0_ast.sashdat

You can export your models from either a) Any modeling node on the Pipelines tab, or b) the Pipeline Comparison tab, to download the champion model.

Pipelines tab

Select any image to see a larger version.
Mobile users: To view the images, select the "Full" version at the bottom of the page.
Pipeline Comparison tab

Let’s walk through this process step-by-step. We’ll download the score code from the pipeline comparison tab in this example.

Five Easy Steps to Create and Download Score Code

Create New Project in Model studio using HMEQ dataset and Advanced template for class target with autotuning. The advanced template will use a number of machine learning methods. From past experience, I know which model will be selected as the champion. The champion model will be gradient boosting, a machine learning model that is actually an ensemble model built from decision trees.
Assign BAD as target and run pipeline.
From Pipeline Comparison tab, select snowman (vertical ellipsis) on the top right, and select Download score code.
This downloads a .zip file to your default downloads folder.
In that .zip file you will find only one file, a dmcas_epscorecode.sas file which references an analytic store, such as _65WHQZZALMID7NXECGXESB9DI_ast.sashdat.

But remember there are TWO parts!

Part 1: Embedded processing score code, dmcas_epscorecode.sas, which

is a .sas file, readable in SAS Studio or, of course, also in SAS 9 BASE SAS
includes data pre-processing instructions, such as imputations and data transformations
includes post-processing instructions, such as reverse transformations

Part 2: The analytic store (astore), _S0MEL0N6NUMBERLETTERC0MB0_ast.sashdat

is a binary file, designed for machine-to-machine communication, not for human readability
is a universal, compact format
contains the state of the model

If you are unfamiliar with an analytic store is, see my previous article. As a reminder state is a copy of all the memory items that are relevant to the next task, for example, scoring. It includes:

The “public” information (i.e., information common to all analytic algorithms). Examples of public information include the list of input variables, the list of output variables, the formats, etc.
The “private” information (i.e., information specific to that particular algorithm, e.g., gradient boosting). Examples of private information include the number of trees, the trees themselves, the scores, etc.

So you may be wondering, where can you find both of these files?

Access and Download the Embedded Processing Score Code and Analytic Store

By default, your dmcas_epscorecode.sas will be downloaded to your local Downloads folder. To look at the dmcas_epscorecode.sas file you can drag it into SAS Studio as shown below.

Or alternatively, you can look at dmcas_epscorecode.sas in SAS 9 in BASE SAS.

And you may ask yourself, “Where is the analytic store?” And you may ask yourself, “How did I get here?” And you may tell yourself, "This is not my beautiful house?" But I digress.

By default the analytics store is in the CAS Models folder.

Let’s use Visual Analytics (“Explore and Visualize Data”) to take a peek and see where it is. We will navigate to Data Sources, cas-shared-default, Models.

We can likewise see it in SAS Environment Manager:

If we sign out and come back in as another user without admin privileges, we can still see the analytic store in the Models file, either SAS Visual Analytics or SAS Environment Manager. And we have the option to load it into memory.

We cannot download it until it is loaded into memory. See how Download table is grayed out in the screen capture below. We can load it directly from Visual Analytics. Or we can load it using SAS Studio and at the same time move it to the Publics folder. We might want to do this, if, for example, the user we want to access it does not have permissions to access the Models folder.

So let’s move it to the Public. In our example, let's give it a new name BethASTORE so we can track what is happening and find it easily, but alternatively we could keep the name the same so that it can be referenced automatically from the dmcas.epscorecode.sas file.

Here is the code to move it to the Publics folder.

Now from the Public folder, anyone can download the analytic store.

We can download as a .csv or a tab delimited file.

Recall that the astore is a binary file decided for machine-to-machine communication. If we download it as a .csv file, we could open it in Notepad. But why?

Hey, Where are the Pre-Processing Steps?

The pre-processing steps are not in the analytic store, they are in the embedded processing score code file, dmcas_epscorecode.sas. To illustrate how the pre-processing step shows up in the dmcas_epscorecode.sas file, let’s compare the downloaded files for two simple pipelines.

a basic gradient boosting pipeline with default pre-processing imputation, to
a gradient boosting pipeline where:

- for DELINQ, imputation is set to 0 for missing
- for VALUE, imputation is set to median and transformation is set to log

Both pipelines:

Use the HMEQ data set
Set BAD as Target, a binary value
Set DELINQ, DEROG and NINQ as interval variables rather than nominal

Download score code for basic pipeline with gradient boosting:

Download score code for the gradient boosting pipeline with additional pre-processing:

Comparing the score we see that the new pre-processing steps are included in the dmcas_epscorecode.sas file.

Once you have created an analytic store, you may wish to score new data in-database.

There is More Than One Way to Skin a Potato

Don’t forget that SAS also has specialized tools to help you. You can use PROC ASTORE to move analytic stores between the client and the server, etc. For example, you can use Model Manager to manage and operationalize models.