BookmarkSubscribeRSS Feed
salammunshi
Calcite | Level 5

Dear all modelling experts,

 

I have built an attrition model by using logistic regression algorithm. Considering r two months operational gap to see the probability of a customer to be churned. In model development process, I'm getting a very good model accuracy with ROC(.94) and Gini(.75).However, while I'm looking at the probability of events category - by looking at P_1.the scores are showing a bit low though the events occurred accurately. Could anyone suggest how could I adjust the probability scores  i.e. if I target top 3 deciles population then I have to consider a customer who has a chance to attrite 30% but how could I reweight that customer as 90%. I have tried to reweighted by AIC or -2logL but still the scores are same. It would be really great to have an expert suggestions.

 

 please find below the average scores distribution from model scores:

 

Analysis Variable : P_1 Predicted Probability: response=1Rank for Variable P_1 N Obs N Mean Std Dev Minimum Maximum
013828138280.0004585410.0001766540.0002530290.000682157
1946894680.00112400.0001181600.0007329030.0011879
211772117720.00145770.0002874580.00121620.0020466
312338123380.00220440.0000608450.00211690.0022530
411304113040.00371330.0008523770.00226550.0053192
511802118020.00872380.00218910.00540450.0131404
611682116820.01507990.00247300.01315580.0232916
711843118430.04112950.01083320.02369940.0622775
810159101590.15002610.07594290.06234670.2864124
913409134090.46778410.17559240.29195470.9407925

 

 I have also provided the model output for your convenience:

 

Model Convergence Status   
Convergence criterion (GCONV=1E-8) satisfied.   
Model Fit Statistics 
CriterionIntercept OnlyIntercept and Covariates 
AIC18844890327.87 
SC188458.3390462.22 
-2 Log L188446.0090301.87 
Testing Global Null Hypothesis: BETA=0
TestChi-SquareDFPr > ChiSq
Likelihood Ratio98144.126712<.0001
Score89795.091312<.0001
Wald32295.41512<.0001

 

Percent Concordant94.3Somers' D0.893
Percent Discordant5Gamma0.9
Percent Tied0.7Tau-a0.222
Pairs6419968407c0.947

 

 

Partition for the Hosmer and Lemeshow Test
GroupTotalresponse = 1response = 0
ObservedExpectedObservedExpected
125145449.512510125135.49
2185363016.711850618519.29
3195575325.881950419531.12
4227966644.572273022751.43
52208984102.852200521986.15
622674340251.132233422422.87
722745802709.692194322035.31
82244528603013.021958519431.98
92097967777559.151420213419.85
10304662196721290.4884999175.52
Hosmer and Lemeshow Goodness-of-Fit   
Test   
Chi-SquareDFPr > ChiSq   
428.92798

<.0001

 

 

 

 

  
1 REPLY 1
salammunshi
Calcite | Level 5

Dear all modelling experts,

 

I have built an attrition model by using logistic regression algorithm. Considering r two months operational gap to see the probability of a customer to be churned. In model development process, I'm getting a very good model accuracy with ROC(.94) and Gini(.75).However, while I'm looking at the probability of events category - by looking at P_1.the scores are showing a bit low though the events occurred accurately. Could anyone suggest how could I adjust the probability scores  i.e. if I target top 3 deciles population then I have to consider a customer who has a chance to attrite 30% at decile 1(top 10% population)  and in actual world that customer is also churning , in that case, how  could I reweight that customer as 90%  at decile 1. I have tried to reweighted by AIC or -2logL but still the scores are same. It would be really great to have an expert suggestions.

 

 please find below the average scores distribution from model scores:

 

Row Labels                      Average of SCORE_1                           Actually closed           # RISK Customer                           Grand

 
00.858566898485165310.029338818
10.602700652345163220.021137116
20.459997375321163430.019641437
30.25116008401166610.024068183
40.13434664523160140.032658923
50.070197285322181750.017716644
60.039646414429172780.024829263
70.03082105356137370.025915411
80.019729811307174770.017565944
90.009666523278153410.018121374
 
 
 

 

Regards,

Mou

 

sas-innovate-2024.png

Available on demand!

Missed SAS Innovate Las Vegas? Watch all the action for free! View the keynotes, general sessions and 22 breakouts on demand.

 

Register now!

How to Concatenate Values

Learn how use the CAT functions in SAS to join values from multiple variables into a single value.

Find more tutorials on the SAS Users YouTube channel.

Click image to register for webinarClick image to register for webinar

Classroom Training Available!

Select SAS Training centers are offering in-person courses. View upcoming courses for:

View all other training opportunities.

Discussion stats
  • 1 reply
  • 337 views
  • 0 likes
  • 1 in conversation