Cumulative best

znhnm — Tue, 07 Mar 2023 15:15:13 GMT

Hi, when I create a visual model in VA, I see "cumulative best" as one of the Lift compenens. What does this tell to us?

Re: Cumulative best

sbxkoenk — Fri, 10 Mar 2023 12:39:09 GMT

Hello,

It's the Cumulative of the BEST-model (see Best-column).

The BEST-model is the model that does perfect prediction.

Suppose you have a binary target (1/0 , Bad/Good , Y/N , ...).

Suppose your target level-of-interest equals "1". In other words, that's the level your model is calculating probabilities for.
The BEST model gives 100% probability to all observed 1's and 0% probability to all observed 0's.
So, if you sort by descending posterior probability for a "1", then all the real 1's are preceding all the real 0's.

So you have observed / real 1's and 0's.
And you have predicted 1's and 0's.
Then you can make the confusion (classification) matrix , and from there you can calculate all kinds of quality statistics (performance stats for the model).

But you are probably looking at LIFT, which you can only measure at a certain depth (@ 5% , @ 10% , ...).

For LIFT, you do not even need a threshold and you do not need to make a 1 / 0 classification (prediction). You only need the posterior probabilities and the real (observed) target value.
You just need to sort by descending posterior probability , and for several depths you can compare the performance of the BASELINE model , the BEST model and your own model(s)!

Koen

topic Cumulative best in SAS Viya

Cumulative best

Re: Cumulative best