Building models with SAS Enterprise Miner, SAS Factory Miner, SAS Visual Data Mining and Machine Learning or just with programming

How do I use the Model Comparison node with the Cutoff node results?

Accepted Solution Solved
Reply
New Contributor
Posts: 4
Accepted Solution

How do I use the Model Comparison node with the Cutoff node results?

Hi,

 

I am trying to use the Cutoff node in SAS Enterprise Miner 14.1 (university "cloud" version) for a logistic regression.  I then want to compare the results of changing the cutoff to the results I get from three other (similar) logistic regressions that don't use the cutoff, just to see the differences in model performance.

 

When I connect my four models (three without the Cutoff and one with) to the Model Comparison node, the results don't seem to capture the model that uses the cutoff.  It produces results, but they appear to be the same as I got before the cutoff (yes, I changed the user input cutoff value from 0.5 to 0.03).

 

Am I not using the Model Comparison correctly with the Cutoff node?  I'm fairly new with SAS, so I'm kinda struggling with this one and the examples I've found seem to stop short of comparing models.

 

Thanks in advance.


Capture.JPG

Accepted Solutions
Solution
‎03-22-2016 06:29 PM
SAS Super FREQ
Posts: 306

Re: How do I use the Model Comparison node with the Cutoff node results?

Posted in reply to cdettmann0

Currently in EM, the Model Comparison node does not use the cutoff from the Cutoff node; however many of the assessment statistics are not affected by the cutoff.  Misclassification rate is of course, but the ASE, lift and gain-related statistics, and KS are not so those wouldn't change based on the choice of cutoff anyway.

View solution in original post


All Replies
New Contributor
Posts: 4

Re: How do I use the Model Comparison node with the Cutoff node results?

Posted in reply to cdettmann0

PS:  I forgot to mention I have a 65% training and a 35% testing set (from the "Data Partition") node.

Solution
‎03-22-2016 06:29 PM
SAS Super FREQ
Posts: 306

Re: How do I use the Model Comparison node with the Cutoff node results?

Posted in reply to cdettmann0

Currently in EM, the Model Comparison node does not use the cutoff from the Cutoff node; however many of the assessment statistics are not affected by the cutoff.  Misclassification rate is of course, but the ASE, lift and gain-related statistics, and KS are not so those wouldn't change based on the choice of cutoff anyway.

New Contributor
Posts: 4

Re: How do I use the Model Comparison node with the Cutoff node results?

[ Edited ]
Posted in reply to WendyCzika

Ok, thanks Wendy, that actually makes sense, I think.  In other words, it's not changing the equation of the predictive line...just the point at which the model chooses to assign a 1 or a 0 to the observation, correct?  I think I just expected to see a separate row in the fit statistics, but I suppose that could be considered redundant.  Appreciate the reply!

☑ This topic is solved.

Need further help from the community? Please ask a new question.

Discussion stats
  • 3 replies
  • 629 views
  • 0 likes
  • 2 in conversation