topic Re: Cumulative Total Expected Profit VS Average Profit in Enterprise Miner in SAS Data Science

Cumulative Total Expected Profit VS Average Profit in Enterprise Miner

andreas_zaras — Wed, 11 Mar 2015 14:34:52 GMT

Hello,

I have created two models in Enterprise Miner i.e. a regression model and a neural network model. I have also input a profit matrix as follows:

Solicit a good customer: A

Solicit a bad customer: B

Ignore a good customer: C

Ignore a bad customer: D

Solicit Ignore

Good A C

Bad B D

The results of the model comparison node (average profit) state that the regression model is better. I guess the formula for concluding to the average profit is: (TP*A+TN*D+FN*C+FP*B)/N. So the regression must have a bigger average profit.

My question is about the additional assessment criteria and more specifically the cumulative total expected profit chart. In this chart, the curve for the neural network is above the curve for the regression in all the deciles of the horizontal axis. How come then regression outperforms neural networls in the average profit measure? Should the two results be aligned?

Thank you very much in advance,

Andreas

Re: Cumulative Total Expected Profit VS Average Profit in Enterprise Miner

mohamed_zaki — Thu, 12 Mar 2015 02:33:04 GMT

Decision Thresholds and Profit Charts

Re: Cumulative Total Expected Profit VS Average Profit in Enterprise Miner

andreas_zaras — Thu, 12 Mar 2015 08:02:54 GMT

Hello,

Thanks for the information. I found the mistake. The model comparison node was optimized for Average Profit/ Loss, whereas the regression and the neural network model were optimized for Average Squared Error. If i change the optimization of the regression and neural network nodes to average profit/ loss the results make sense.

Thanks again,

Andreas

Re: Cumulative Total Expected Profit VS Average Profit in Enterprise Miner

mohamed_zaki — Thu, 12 Mar 2015 12:32:12 GMT

I do not think that what you concluded is right. The profit chart and average profit is different. The regression being first in average profit does not mean that it should came above the neural in the chart.