Hi,
I have built a logistic regression model to predict customers who are more likely to click on a marketing offer. My model is not great as I have a AUC=0.58 and 50% capture only 60%. So I have built the model on a training sample (9 months) then kept 2 months of data as sample held out from model dev (Validation) to test my prediction. Once I finished to build the model on the training, I have scored the validation (2 months dataset) then sorted the scored validation, and sliced the target variable and probabilities into deciles ..like below...It doesn't seem that my model is ranking well ...Is it because the AUC is low..so the predictive power is not great? Your help will be much appreciated....Many Thanks
proc rank data=VALIDATION (KEEP=CLICK_FLAG EM_EVENTPROBABILITY) groups=10 out=pred_valid;
var EM_EVENTPROBABILITY; ranks decile;
run;
data pred_valid;
set pred_valid;
decile=decile+1;
run;
proc sort data=pred_valid;
by descending EM_EVENTPROBABILITY;
run;
proc means data=pred_valid n mean sum;
var CLICK_FLAG;
class decile;
run;
proc means data=pred_validn mean sum;
var EM_EVENTPROBABILITY;
class decile;
run;
| Decile | Training Response Rate | Training Mean_Score | Validation Response Rate | Validation Mean_Score |
| 10 | 10% | 0.10 | 6% | 0.10 |
| 9 | 9% | 0.08 | 6% | 0.08 |
| 8 | 7% | 0.07 | 7% | 0.07 |
| 7 | 6% | 0.06 | 7% | 0.06 |
| 6 | 6% | 0.06 | 7% | 0.06 |
| 5 | 6% | 0.06 | 7% | 0.06 |
| 4 | 6% | 0.05 | 7% | 0.05 |
| 3 | 5% | 0.05 | 6% | 0.05 |
| 2 | 5% | 0.05 | 5% | 0.05 |
| 1 | 4% | 0.03 | 7% | 0.03 |
