Machine Learning, Model Management, Governance & Workflows with SAS Customer Intelligence 360

[#1]

Today’s machine learning techniques allow analysts to quickly train and create more models faster than ever. As efficiency increases, authoring models is only one aspect of the analytical lifecycle that brands need to consider. As the number of models increases to support more business objectives, so does the requirement to manage these assets as valuable competitive differentiators.

Figure 1: Model management as a business process

Model management is not a one-time activity, but an essential business process. Models must be well developed and validated to demonstrate that they are working as expected. Outcome analysis is necessary to:

• Ensure that the scores derived from applying the model to new data are accurate.
• Verify that model performance over time remains satisfactory.

Other aspects include cataloging and tracking this growing inventory of analytical assets, while providing support for the governance of these models using version control through repeatable and traceable workflows.

Practical considerations for data science emerge when an analysis worthy of addressing your marketing team’s business problem pivots the work stream to taking action via model deployment. Options within SAS include:

• Publishing of models for scoring directly in database. 
• Score holdout data.
• Import scoring of other models to compare.
• Download score code in multiple languages.
• Download scoring API for invoking models as web services.

Users can store models in a common model repository and organize them within projects and folders. A project consists of the models, attributes, tests and other resources that can be used to:

• Evaluate models for champion model selection.
• Monitor model performance to minimize exposure to decaying predictions.
• Publish models to other areas of the SAS platform or third-party technologies.

To ensure that a champion model in a production environment is performing efficiently, users can collect performance data that has been created by the model at intervals that are determined by your brand. Performance data is used to assess model prediction accuracy. For example, users might want to assess performance weekly, monthly or quarterly. Monitoring can be performed on champion and challenger models, and as data trends change over time, the champion model can be improved by:

• Replacement by a challenger (another algorithm within the project starts fitting the data more accurately).
• Tuning or refitting the model performed by an analyst.

SAS Customer Intelligence 360 enables brands to use first-party data to make better decisions using predictive analytics and machine learning in conjunction with business rules across a hub of channel touch points. As your journey into analytical marketing use cases progresses, usage of your modeling intellectual property cannot be under-exploited. It’s competitive differentiation awaiting to be deployed.

With that said, I invite you to view a video and technology demonstration that will address the following topics:

1. What capabilities are available to monitor model efficacy?

2. How are users alerted to model performance degradation?

3. When and how are decaying models retrained or replaced by new models?

Learn more about how the SAS platform can be applied for marketing data management here. In addition, I strongly recommend reviewing @dishaw's recent SAS Communities article entitled "Organize and Manage All Types of Analytic Models and Pipelines."