BookmarkSubscribeRSS Feed
Kanyange
Fluorite | Level 6

Hi,

I have built a logistic regression model to predict customers who are more likely to click on a marketing offer. My model is not great as I have a AUC=0.58 and 50% capture only 60%. So I have built the model on a training sample  (9 months) then kept 2 months of data as sample held out from model dev (Validation) to test my prediction. Once I finished to build the model on the training, I have scored the validation (2 months dataset) then sorted the scored validation, and sliced the target variable and probabilities into deciles ..like below...It doesn't seem that my model is ranking well ...Is it because the AUC is low..so the predictive power is not great?  Your help will be much appreciated....Many Thanks

proc rank data=VALIDATION (KEEP=CLICK_FLAG EM_EVENTPROBABILITY) groups=10 out=pred_valid;

  var EM_EVENTPROBABILITY; ranks decile;

run;

data pred_valid;

  set pred_valid;

decile=decile+1;

run;

proc sort data=pred_valid;

by descending EM_EVENTPROBABILITY;

run;

proc means data=pred_valid n mean sum;

  var CLICK_FLAG;

  class decile;

run;

proc means data=pred_validn mean sum;

var EM_EVENTPROBABILITY;

class decile;

run;

DecileTraining Response RateTraining Mean_ScoreValidation Response RateValidation Mean_Score
1010%0.106%0.10
99%0.086%0.08
87%0.077%0.07
76%0.067%0.06
66%0.067%0.06
56%0.067%0.06
46%0.057%0.05
35%0.056%0.05
25%0.055%0.05
14%0.037%0.03
3 REPLIES 3
jwexler
SAS Employee

Kanyange, would you mind sharing your modeling code?  It would help to understand the options you selected, along with a snapshot of the output.

Thanks,

Jonathan

Kanyange
Fluorite | Level 6

Hi Jonathan,

Thank you for your reply,...yes sure , I have attached the Code and the output...Please let me know if you can open the attachement,....

Cheers

jwexler
SAS Employee

Kanyange, this question would be best posted in the SAS Stat Community

Given your needs with PROC LOGISTIC, they would be best to respond appropriately.

Let me know if you need any more help.

Thanks,

Jonathan

sas-innovate-2024.png

Join us for SAS Innovate April 16-19 at the Aria in Las Vegas. Bring the team and save big with our group pricing for a limited time only.

Pre-conference courses and tutorials are filling up fast and are always a sellout. Register today to reserve your seat.

 

Register now!

How to choose a machine learning algorithm

Use this tutorial as a handy guide to weigh the pros and cons of these commonly used machine learning algorithms.

Find more tutorials on the SAS Users YouTube channel.

Discussion stats
  • 3 replies
  • 1218 views
  • 0 likes
  • 2 in conversation