Versioning in SAS Model Manager

3 Likes

SAS Model Manager supports both model and project versioning to help track changes and preserve model snapshots across important milestones. Versioning is a key capability that supports proper model governance and validation processes. For today’s article, let’s review how to use project and model versioning as well as discuss recent enhancements to versioning.

Project Versioning

Projects hold multiple models and the information specific to one analytical use case. Projects can be versioned within SAS Model Manager, but project versioning has a few key differences from model versioning. For example, each project version holds a distinct set of models. Commonly, a new project version is created whenever the training data for the set of models updates, such as during model retraining. In fact, whenever a user opts to retrain their modeling projects from SAS Model Manager, a new project version is generated automatically.

Project versions can be created, deleted, renamed, and viewed from the Version drop-down in the Models tab of the project. This short video demonstrates how to interact with project versions:

(view in My Videos)

Model Versioning

Each version of a model holds a variety of files and information, including the score code, model ASTORE or pickle file, properties, link to the training table, requirements, and more.

From the Versions tab of the model, users can view each version of the model as well as create a new model version. This short video demonstrates how to interact with model versions:

(view in My Videos)

As of writing just after the release of SAS Viya 2024.01, a new model version is generated by the following activities:

Publishing a model.
Setting a model as champion.
Making changes to the original model within SAS Model Studio and re-registering the model.
Manually creating a new model version.

Whenever a new model version is created, the previous model versions are locked from editing. Locking the previous model versions from editing ensures that the files and information we have within SAS Model Manager accurately reflects what is available in deployment destinations.

Users have provided several pieces of feedback for how we can improve versioning to better fit their needs. We have already released several new capabilities based upon that feedback and are currently working on addressing more of that feedback. So next, let’s review what’s new and on the horizon for versioning within SAS Model Manager.

What’s New with Versioning in SAS Model Manager

Prior to our recent changes, users were restricted to only use the latest version of a model for decision flows, score testing, publishing, and performance monitoring. After discussions with our users, we decided to remove that restriction. Allowing users to leverage previous model versions provides greater flexibility in model validation and governance processes. Let’s dive into how you can start using each of these new capabilities.

Use Any Model Version within a Decision Flow

Since SAS Viya 2023.05, SAS Intelligent Decisioning users have been able to specify which version of a model is used within a decision flow. After a model is added to the flow, users can select the model version from the properties pane:

This allows users to create decision flows using a specific model version or opt to always pick up the latest version of a model. Multiple versions of a model can exist within the same decision flow, enabling A/B Testing of new model versions against previous ones.

Score Test All Versions of a Model

With the release of SAS Viya 2023.09, any version of a model can be selected for a score test. After users select their model inside a scoring test, they can then select which version of that model they would like to use. Users have two options for selecting a model version to use for score testing:

Select a model version by number: this option will create a scoring test that will always use the selected version of the model.
Select to use latest model version: this option will always use the latest version of a model. If a new version of a model becomes available, re-running this test will use the newest version of a model.

These two options give users a static option and a dynamic option for creating scoring tests.

Creating scoring tests for previous model versions allows users to continually test their production and pre-production models and provide stronger model validation over time.

To learn more about this new capability, see the following demo:

(view in My Videos)

Publish All Versions of a Model

From SAS Viya 2023.10 onwards, any version of a model can be published. From the publishing dialogue, users can select which version of a model they would like to publish. This means that the same version of a model can be deployed in multiple destinations, such as a testing environment and a production environment. This also allows for easy roll-back of a production model to a previously deployed version.

To learn more about this new capability, see the following demo:

(view in My Videos)

Monitor All Versions of a Model

Beginning on SAS Viya 2024.01, any version of a model can be monitored. Like score testing, users have two options for selecting a model version to monitor:

Select a model version by number: this option will always monitor the selected version of the model.
Select to use latest model version: this option will always monitor the latest version of a model. If a new version of a model becomes available, re-running the performance monitoring report will use the newest version of a model.

Monitoring the performance of a model across versions can help users react to decay if multiple versions of a model are deployed across destinations or determine if a production model should be rolled-back to a previous version.

To learn more about this new capability, see the following demo:

(view in My Videos)

Upcoming Version Enhancements

Our users have reported that they would like automatic version incrementing (also know as version bumping) when publishing a model or marking a model as champion to be optional. This is the next piece of feedback we would like to address for versioning enhancements. It is necessary to continue to lock models from editing when publishing or marking a model as champion to ensure models within SAS Model Manager match what is available in external destinations and environments, but we can let the user decide when to create a new model version.

Versioning is a key component for robust model governance and management. Based on user feedback, we have made adjustments to better support their processes and use cases. Interested in providing your own feedback to the SAS Model Manager team? SAS Product Suggestions is a board for product feedback and ideas.

Want to learn more about SAS Model Manager? Then check out the following resources: