Definition of Lift in Visual Statistics and Proc IMSTAT

[pvareschi]

Re: Predicting Analytics on Big Data (PABD)

I might be wrong but it looks to me the definition and calculation of lift (for the Lift Chart) used in Visual Analytics and Proc IMSTAT are different from what is presented in "Applied Analytics Using SAS Enterprise Miner (AAUEM)" and "Predictive Modeling Using Logistic Regression (PMULR)".

At page 3.15 of the course text of PABD, lift is defined in terms of "percent of captured responses", which is confirmed by the figures in table shown at page 3.94.

However, in AAUEM (page 6-17 and 6-20) and PMLUR (page 4.11), lift is defined in terms of "response rate" (i.e. Positive Predictive Value PV+), which was confirmed in the reply to a previous post.

The two concepts are not equivalent: which one is correct?

[ed_sas_member · Accepted solution]

Hi @pvareschi 

To me, the lift at a given depth is the ratio of the performance of a classifier (-> your model) to the performance obtained by chance:

in other words, at a specific depth, the classifier will obtain "lift" times as many events (binary response = 1) as that obtained by random sampling.

So lift = PV+ / p

where p is the % of event in the population

[pvareschi]

Thank you for your clarification; I agree with your view and meaning of lift.

I just wanted to emphasize that I am not sure it is appropriate to refer to "captured response": based on PV+, lift measures, at a given depth, how much better a model is in increasing the response rate compared to a random approach; so the emphasis is on "response rate" rather than "captured response"

[ed_sas_member]

Hi @pvareschi 

Yes I agree with you

Best,

[pvareschi]

Thank you 🙂

In that case, may I suggest that someone responsible for the training and exam take a look at how questions are phrased (and correct solution given) for the "SAS Certified Specialist: Advanced Predictive Modeling" certification?

Saying that because I have just completed the practice exam "SAS Advanced Predictive Modeling (PE-225P)" and one of the question I got wrong was exactly on the definition of lift: the model answer provided was based on "captured response rate" whereas I gave "response rate", which, I believe, should be the correct answer!