This post is co-authored with my colleague Ivan Nardini
Machine learning adoption in business has increased hugely over the past years. However, many organizations find it difficult to get a return on their investment from Data Science initiatives. This is because of the operationalization barrier, the so-called ‘last mile problem’, and lack of efficient model governance across the whole model lifecycle. The abundance of separate development and deployment frameworks provides an additional challenge in scaling analytics across the whole company. This is where a ModelOps approach comes into play. It streamlines work across the entire model lifecycle- from experimentation to execution.
This article provides an example of the ModelOps process created for the large European Oil and Gas company. It shows how SAS can help operationalize the model on the customer’s environment of choice using SAS Model Manager and SAS Workflow Manager.
In this process, we have 3 different environments:
Figure 1. Architecture of the ModelOps process
All these environments could work in the cloud, leveraging the benefits of this technology, and may utilize most of the popular tools for the development and deployment of machine learning models.
The process runs according the following stages:
SAS Viya acts as the key enabler for this ModelOps process because of two modules:
This process is reflected in SAS Workflow Manager’s workflow definition.
Figure 2. ModelOps process represented in BPMN format of the workflow definition
SAS Workflow Manager has a simple and flexible graphical representation of the operationalization process. Using the graphical interface, you can create, modify and track your ModelOps processes. It aligns with widely used Business Process Management Notation v 2.0. Therefore, you can use directed acrylic graphs (DAGs), and also run cycled processes, waiting points, timers, branches and so on.
The key components of the workflow process are:
The deployment stage is the first part of the workflow, in which the model is moved into production on OpenShift Kubernetes.
Figure 3. User interface to work with workflow and service task at SAS Viya platform
At the starting point, this workflow process receives all necessary parameters, such as name or unique identifier of the SAS Model Manager’s project (to refer to the right model content), communication channel for notifications from the workflow engine (in our example, – a mailing service and Microsoft Teams) and KPI for defining acceptable model quality threshold (in our case measured in Kolmogorov-Smirnov statistic).
The next stage is the ‘approve champion model’ user task where the validator assesses the model before deployment into production. This is an example of combining human expertise with business rules and running within the single process flow. According to this task, the user should set a certain tag of the image, as a prerequisite for the next stage.
Figure 4. We receive Microsoft Teams notification of model deployment status as workflow process goes by
At the end of the deployment stage, our model is up and running and can be instantly used to support decision making.
Figure 5. Scoring process of Tensorflow model on OpenShift.
As soon as the model is deployed, the workflow moves to "Production stage" sub-process. This covers Performance monitoring and retraining of the champion model deployed on OpenShift.
Figure 6. Detailed view of the ‘production stage’ subprocess
During the production stage, all data processed by the model are saved. Later on, this data will be useful to assess model performance and for retraining purposes.
We run repetitive assessments of model performance, using a defined timer object. We can set a flexible schedule for testing, for example: once per day, week or month, depending on the business case.
If model performance results will be unsatisfactory (in our case measured in terms of KS index), the process will trigger automatic retraining of the Tensorflow estimator using the new data. It will also register the new version of the model with the PZMM and SASCTL library.
Then the new model will be validated manually in the ‘approve champion model’ user task and the process will go on into ModelOps infinite loop until the business problem remains relevant.
This shows how SAS can help Data Scientists to operationalize their models in a code-agnostic way. SAS Model Manager and SAS Workflow Manager enable a ModelOps process for machine learning development and deployment frameworks-of-choice. As result we reduce time-to-market of analytical models and make their performance stabile over time. It is possible because of the openness of the SAS Viya platform, reflected in the wide range of built-in RESTs collection and advanced orchestration capabilities.
Thanks for reading. Your feedback and questions are welcomed!
Code repository of the mentioned ModelOPS scenario
Organize and manage all types of analytic models and pipelines by Diana Shaw
Model Validation series blog by Tamara Fischer and Hans-Joachim Edert
Open Source Model Management through REST APIs: Registration by Yi Jian Ching
Mastering the Model Lifecycle Orchestration Journey guide by Marinela Profi
Are you ready for the spotlight? We're accepting content ideas for SAS Innovate 2025 to be held May 6-9 in Orlando, FL. The call is open until September 25. Read more here about why you should contribute and what is in it for you!
Data Literacy is for all, even absolute beginners. Jump on board with this free e-learning and boost your career prospects.