Easy Deployment and Monitoring for Multinomial Classification Models

[#1]

Many problems in analytics and data science are often solved using a binary classification model. A binary classification model has two possible outcomes, such as Yes or No and True or False. But not all problems fit neatly into two categories.

In fact, the popular (and not so cool anymore let's face it) Iris Data Set features three possible outcomes: one for each Iris species in the sample. And of course, there are prediction models as well, whose predictions fall along a continuum.  

SAS Model Manager fully supports binary classification and prediction models already. But with the release of SAS Model Manager 2021.2.6, we’ve baked-in broader support for multinomial classification models. This includes metadata support for target event values and performance monitoring for multinomial classification models. The Project Properties pane in SAS Model Manager lists the possible event values for multinomial models and can automatically discover those values.  

Additionally, Performance Monitoring includes charts and metrics that are specific to Multinomial models.  

Learn more about our new features for multinomial models in the following video: 

We are continuing to expand our support for various types of models. What kinds of model support would you like to see within Model Manager? Leave your comment below!  

[sbxkoenk]

Hello @MarinelaProfi ,

Is there anything in Model Manager for models with only input variables (unsupervised learning) and no target / outcome? 

Suppose you have a clustering solution.

Every night you score new observations which you add to the existing clusters.

You could follow up in time whether the clusters remain as well separated from each other (as during training time) and whether there are perhaps observations popping up that should not be assigned to a cluster (because they are too far away from each cluster).

Suppose you have an anomaly detection solution (multivariate outlier detection).
Every night you score new observations.
Maybe the n° of anomalies goes up over time (?).
Maybe you need to re-train your anomaly detection model after a few months because things that were outlying ( compared to the observations of ) 3 months ago are no longer exceptional?

Thanks,

Koen

[SophiaRowland]

Hello @sbxkoenk!

It sounds like you are looking for performance monitoring of unsupervised models. We currently do not support performance monitoring of unsupervised models out-of-the box, but that is a great idea for a future enhancement! Just to note, our performance monitoring reports can be extended using SAS Visual Analytics charts and custom code, so that is a potential route you can take to extend our reports to monitor your models today. I have your suggestion noted in our internal systems, but I suggest using SASware Ballot Ideas with the SAS Model Manager tag if you want to keep track of your feature idea. 

Thanks, 
Sophia Rowland