Lift chart value in business case after Random Forest model

Fischer03 — Fri, 19 Oct 2018 11:35:50 GMT

Hello,

I have a question regarding the lift value that I shoud expect on the first decile derived from a Random Forest Models.

In details, I computed a HP random Forest model using SAS Miner to identify the propensity to buy a product. As result, I have found that my model produce a lift chart value about 9 (on a first decile) in the validation set.

In your opinion and experiences, isn't that value too high? Should I expect a smaller value?

As each of the trainig and validation lift chart are the same, I tend to exclude overfitting; is there somthing else should I controll to be sure my model is well done?

Thank you very much in advance!

Re: Lift chart value in business case after Random Forest model

DougWielenga — Wed, 16 Jan 2019 18:37:32 GMT

I computed a HP random Forest model using SAS Miner to identify the propensity to buy a product. As result, I have found that my model produce a lift chart value about 9 (on a first decile) in the validation set. In your opinion and experiences, isn't that value too high? Should I expect a smaller value?

The maximum possible lift is relative to the overall population rate of your event. The lift in the first decile will differ across data even if they have the same overall population rate since the ability of the available data to predict an event varies across data. Lift is useful since it has no units -- it is just a relative increase/decrease in the occurence of an event, but this almost means that you must consider both the lift and the actual predicted occurrence to properly assess how "big" it is. For a discussion of how the occurrence rate and lift are related, see the discussion at

https://communities.sas.com/t5/SAS-Data-Mining-and-Machine/Help-with-over-under-sampling-of-the-rare-event-in-predictive/m-p/388519#M5851

Hope this helps!

Doug

topic Lift chart value in business case after Random Forest model in SAS Data Science

Lift chart value in business case after Random Forest model

Re: Lift chart value in business case after Random Forest model