ModelOps, MLOps, and Finding Value in Analytics
In a recent survey, 42% of data scientists reported that their results were not used by decision makers. This is a twofold issue. First, there are the organizations that have invested hundreds of billions of dollars into analytics worldwide annually. Organizations have invested in data collection, cleaning, storage, and preparation. Data scientists have used that data to find insights, uncover patterns, and build models. When this work goes unused, it is demotivating. Ultimately, data scientists want to apply what the models they have developed to solve problems and make better decisions. Data scientists want to make an impact. But despite all this work and investment, there is still a gap between finding insights and using insights. Enter MLOps and ModelOps.
The differences and definitions of MLOps and ModelOps vary. Some definitions of ModelOps say that it is focused on the operationalization of all models of which MLOps is as subset solely focused on Machine Learning models. Others recognize the definition of ModelOps has evolved to be an enterprise capability that involves diverse groups across an organization. The real difference between MLOps and ModelOps lies in their intentions.
Image from ML Ops: Machine Learning as an Engineering Discipline
MLOps is the application of DevOps principles for Machine Learning models. DevOps bridges the gap between IT and software development to create robust processes around operationalizing software. Thus, MLOps has been spearheaded by IT and software engineering skillsets. The focus of MLOps has been on creating repeatable and standardized processes for the usage and maintenance of machine learning models. And this is a worthy pursuit. By putting models into production faster and by actively monitoring performance of those models, organizations have the technical ability to use machine learning models. But without wider involvement from the business, MLOps can fall short.
A data scientist must wear many hats. They need to not only understand the statistical methods they employ and the data they are using, but they need to understand the problem they are solving and the business impact their work will produce. Their work does not exist in a vacuum. The models that data scientists build have context. MLOps can fall short by ignoring that context. Models cannot be treated simply as source code.
ModelOps is in direct opposition to the idea of a data scientist throwing their model over the fence to IT. ModelOps is an enterprise effort for improved decision-making. It includes the decision makers understanding the model’s output and how it fits into their processes. For routine and operational decisions, decision makers may create standardized decisioning flows around how the output of the model is handled. This also includes end-users being able to ask why a model made a specific prediction. Model risk teams will be involved as well, ensuring that risks of using the model are mitigated and controlled. Of course, data scientists have a role as well. Data Scientists ensure that the model remains accurate and usable, making adjustments as necessary. IT and engineers will continue to work getting models into production in a timely manner as well as maintaining the infrastructure models need to run. ModelOps is an enterprise effort for improved decision making through analytics.
Image from The 4 Trends That Prevail on the Gartner Hype Cycle for AI, 2021
ModelOps continues to innovate. Gartner recently called out model operationalization as a top trend on their hype cycle and predicts rapid growth and maturity in this area. But in the end, MLOps and ModelOps are just terms to describe a set of best practices and frameworks. How these terms are defined and differentiated may continue to evolve. So, let’s focus on our intentions. Operationalizing models cannot be a siloed effort between data scientists and IT. Multiple groups should be involved to ensure models and analytics are used effectively for quality decision making.
When selecting a partner for operationalizing models, organizations find a multitude of companies within the MLOps and ModelOps space. It is critical for organizations to sort through the barrage of established and emerging companies to find a partner that can help them gain value from analytics. SAS Model Manager includes ModelOps best practices baked-in from an easy-to-use interface with extendibility and flexibility to add customizations to fit specific organizational needs. Georgia Pacific uses SAS Model Manager to monitor and deploy over 1900 models developed using both SAS and open source languages. Georgia Pacific was able to make ModelOps accessible across their organization. More about their story is available here. S-Bank, Finland’s top retail bank, leverages SAS Model Manager to achieve better alignment between the analytics team, business development team, and sales team, which has allowed them to create solutions to provide better customer service and more accurate loan processing times. More about their story is available here. Choosing a reliable and experienced partner for your ModelOps journey can offer the tooling and expertise necessary to turn data into insights and improve decision making through analytics.
