Benchmark Score to include in Model Comparison

SCoop — Sun, 04 Jun 2017 13:46:08 GMT

I'm developing a credit score. Let's say I have 100 variables plus a Target. I also have a benchmark score (the current score being used) that I'm trying to out-perform in model development. Using Enterprise Miner, how can I use this benchmark score in the Model Comparison Node? I do not want this score to become an input into a model but I would like to see it's performance in Model Comparison (eg ROC curve). I've experimented with Meta Data roles (eg Target,Prediction) but I can't seem to get it right.

Re: Benchmark Score to include in Model Comparison

WendyCzika — Mon, 05 Jun 2017 13:11:00 GMT

The Model Import node on the Model tab should be able to help you, here is part of the Reference Help for that node...

Defining the Scored Data Set in the Input Data Source Node

You can use the Input Data Source node to represent a scored data set that contains predicted values for the target variable. The scored data set can then be passed to the Model Import node for assessment in the Model Manager or in the Model Comparison node.

A SAS modeling procedure, such as PROC LOGISTIC, or a customized data set can be used to create the scored data set.

This post might help as well:

https://communities.sas.com/t5/SAS-Data-Mining-and-Machine/Help-with-Model-Import-in-SAS-EM/m-p/192154/highlight/true#M2415

topic Re: Benchmark Score to include in Model Comparison in SAS Data Science

Benchmark Score to include in Model Comparison

Re: Benchmark Score to include in Model Comparison

Defining the Scored Data Set in the Input Data Source Node

You can use the Input Data Source node to represent a scored data set that contains predicted values for the target variable. The scored data set can then be passed to the Model Import node for assessment in the Model Manager or in the Model Comparison node.