Before publishing models to a production environment, organizations often have a process to validate the model’s effectiveness as well as ensure the model is in good standing. Having a robust process to validate a model prior to production helps prevent production errors, poor decision making, and even disparate outcomes due to bias. But how can organizations prevent unauthorized models from going to production and ensure that their processes are followed. In this article, I’ve detailed three ways using SAS Viya 4 to prevent unauthorized model publishing.
SAS Model Risk Management
SAS Model Risk Management (MRM) is a tool on SAS Viya for documenting and mitigating model risk. SAS Model Risk Management integrates directly with SAS Model Manager to create a seamless flow of information between the two tools. Whereas SAS Model Manager is used by data scientists and MLOps engineers to manage, deploy, and monitor their models, MRM is used by risk analysts and risk teams. These risk teams often have their own processes and tests that depend on information provided by data scientists and MLOps engineers. Not to worry, the integration between MRM and Model Manager on SAS Viya help ensure the various teams involved can efficiently work together.
Models can be linked between Model Manager and MRM. This linkage tells risk teams which models they need to audit as well as allows for information to be shared between teams. Once a model is linked within MRM, that model can not be published from Model Manager until it has the appropriate risk sign-off and approval. The demo below walks through how to link models within Model Manager and MRM as well as the publishing restriction outcome.
User Permissions
Administrators can prevent publishing for a user or a group by creating a prohibit rule for ModelPublish/models. Users can still see models within the model repository, but they will get an error when they try to publish the model. This method can be used in conjunction with a custom workflow to create a standardized flow for validation and approval. SAS Workflow Manager is included with SAS Model Manager. Workflows designed within SAS Workflow Manager represent business processes. These workflows can be designed to include manual tasks, performed by a human, as well as automated tasks, logic gates, timers, and more. The demo below walks through how to restrict publishing for a specific user but also shows how workflows can be used to publish a model after the appropriate approvals are received.
Model Locks
Instead of preventing a user or group from publishing any model, you can also lock models from being published by anyone. No one will be able to publish a model until after the lock is removed from the model. Model locking and unlocking can be performed using REST APIs, which means they can easily be pulled into a workflow. The demo below walks through a workflow where a user selects which model to lock from publishing with the lock being removed after proper approval is achieved. This workflow is a simple example and can be extended to support a robust approval process.
Want to learn more?
Analyst firm IDC recently praised SAS Model Manager for our strong governance and production capabilities. To learn why, check out these resources!
