topic Re: Definition of Lift in Visual Statistics and Proc IMSTAT in SAS Academy for Data Science

Definition of Lift in Visual Statistics and Proc IMSTAT

pvareschi — Fri, 29 May 2020 07:40:43 GMT

Re: Predicting Analytics on Big Data (PABD)

I might be wrong but it looks to me the definition and calculation of lift (for the Lift Chart) used in Visual Analytics and Proc IMSTAT are different from what is presented in "Applied Analytics Using SAS Enterprise Miner (AAUEM)" and "Predictive Modeling Using Logistic Regression (PMULR)".

At page 3.15 of the course text of PABD, lift is defined in terms of "percent of captured responses", which is confirmed by the figures in table shown at page 3.94.

However, in AAUEM (page 6-17 and 6-20) and PMLUR (page 4.11), lift is defined in terms of "response rate" (i.e. Positive Predictive Value PV+), which was confirmed in the reply to a previous post.

The two concepts are not equivalent: which one is correct?

Re: Definition of Lift in Visual Statistics and Proc IMSTAT

ed_sas_member — Fri, 29 May 2020 09:28:26 GMT

Hi @pvareschi

To me, the lift at a given depth is the ratio of the performance of a classifier (-> your model) to the performance obtained by chance:

in other words, at a specific depth, the classifier will obtain "lift" times as many events (binary response = 1) as that obtained by random sampling.

So lift = PV+ / p

where p is the % of event in the population

Re: Definition of Lift in Visual Statistics and Proc IMSTAT

pvareschi — Fri, 29 May 2020 10:02:17 GMT

Thank you for your clarification; I agree with your view and meaning of lift.

I just wanted to emphasize that I am not sure it is appropriate to refer to "captured response": based on PV+, lift measures, at a given depth, how much better a model is in increasing the response rate compared to a random approach; so the emphasis is on "response rate" rather than "captured response"

Re: Definition of Lift in Visual Statistics and Proc IMSTAT

ed_sas_member — Fri, 29 May 2020 10:04:21 GMT

Hi @pvareschi

Yes I agree with you

Best,

Re: Definition of Lift in Visual Statistics and Proc IMSTAT

pvareschi — Fri, 29 May 2020 10:12:06 GMT

Thank you 🙂

In that case, may I suggest that someone responsible for the training and exam take a look at how questions are phrased (and correct solution given) for the "SAS Certified Specialist: Advanced Predictive Modeling" certification?

Saying that because I have just completed the practice exam "SAS Advanced Predictive Modeling (PE-225P)" and one of the question I got wrong was exactly on the definition of lift: the model answer provided was based on "captured response rate" whereas I gave "response rate", which, I believe, should be the correct answer!