BookmarkSubscribeRSS Feed
znhnm
Quartz | Level 8

Hi, when I create a visual model in VA, I see "cumulative best" as one of the Lift compenens. What does this tell to us?

 

znhnm_0-1678202153134.png

 

1 REPLY 1
sbxkoenk
SAS Super FREQ

Hello,

 

It's the Cumulative of the BEST-model (see Best-column).

The BEST-model is the model that does perfect prediction.

 

Suppose you have a binary target (1/0 , Bad/Good , Y/N , ...).

Suppose your target level-of-interest equals "1". In other words, that's the level your model is calculating probabilities for.
The BEST model gives 100% probability to all observed 1's and 0% probability to all observed 0's.
So, if you sort by descending posterior probability for a "1", then all the real 1's are preceding all the real 0's.

 

So you have observed / real 1's and 0's.
And you have predicted 1's and 0's.
Then you can make the confusion (classification) matrix , and from there you can calculate all kinds of quality statistics (performance stats for the model).

 

But you are probably looking at LIFT, which you can only measure at a certain depth (@ 5% , @ 10% , ...).

For LIFT, you do not even need a threshold and you do not need to make a 1 / 0 classification (prediction). You only need the posterior probabilities and the real (observed) target value.
You just need to sort by descending posterior probability , and for several depths you can compare the performance of the BASELINE model , the BEST model and your own model(s)!

 

Koen