Building models with SAS Enterprise Miner, SAS Factory Miner, SAS Visual Data Mining and Machine Learning or just with programming

Cumulative Total Expected Profit VS Average Profit in Enterprise Miner

Accepted Solution Solved
Reply
Frequent Contributor
Posts: 75
Accepted Solution

Cumulative Total Expected Profit VS Average Profit in Enterprise Miner

Hello,

I have created two models in Enterprise Miner i.e. a regression model and a neural network model. I have also input a profit matrix as follows:

Solicit a good customer: A

Solicit a bad customer: B

Ignore a good customer: C

Ignore a bad customer: D

            Solicit  Ignore

Good      A          C

Bad        B          D

The results of the model comparison node (average profit) state that the regression model is better. I guess the formula for concluding to the average profit is: (TP*A+TN*D+FN*C+FP*B)/N. So the regression must have a bigger average profit.

My question is about the additional assessment criteria and more specifically the cumulative total expected profit chart. In this chart, the curve for the neural network is above the curve for the regression in all the deciles of the horizontal axis. How come then regression outperforms neural networls in the average profit measure? Should the two results be aligned?

Thank you very much in advance,

Andreas


Accepted Solutions
Solution
‎03-12-2015 04:02 AM
Frequent Contributor
Posts: 75

Re: Cumulative Total Expected Profit VS Average Profit in Enterprise Miner

Hello,

Thanks for the information. I found the mistake. The model comparison node was optimized for Average Profit/ Loss, whereas the regression and the neural network model were optimized for Average Squared Error. If i change the optimization of the regression and neural network nodes to average profit/ loss the results make sense.

Thanks again,

Andreas

View solution in original post


All Replies
Super Contributor
Posts: 490

Re: Cumulative Total Expected Profit VS Average Profit in Enterprise Miner

Solution
‎03-12-2015 04:02 AM
Frequent Contributor
Posts: 75

Re: Cumulative Total Expected Profit VS Average Profit in Enterprise Miner

Hello,

Thanks for the information. I found the mistake. The model comparison node was optimized for Average Profit/ Loss, whereas the regression and the neural network model were optimized for Average Squared Error. If i change the optimization of the regression and neural network nodes to average profit/ loss the results make sense.

Thanks again,

Andreas

Super Contributor
Posts: 490

Re: Cumulative Total Expected Profit VS Average Profit in Enterprise Miner

I do not think that what you concluded is right. The profit chart and average profit is different. The regression being first in average profit does not mean that it should came above the neural in the chart.

☑ This topic is SOLVED.

Need further help from the community? Please ask a new question.

Discussion stats
  • 3 replies
  • 928 views
  • 1 like
  • 2 in conversation