11-24-2017 08:38 AM - edited 11-24-2017 09:22 AM
My goal is to get a new scorecard better than the old one. But on test month it has higher bad rate and lower approval...
What am I doing wrong?
I try to build a scorecard model in Enterprise Miner:
I keep the only non-correlating predictors:
It seems my variables are pretty valuable:
And I meet several problems:
1) very few target events:
to overcome this limitaion I involved frequency variable, but I suspect model to bias:
2) cannot get stable model
3) train / validation gini varies drastically: before reject inference, data partition 50/50 stratified, train gini=0.52, validation gini=0.49
And some questions:
1) How to estimate bad rate and approval of scorecard model?
All I need - is to improove old scorecard model, to archive this I tried to exclude predictors (start from the lowest information value) and add new ones (with high IV) . Honestly speaking have no other ides of doing that.
P.s. I used Naim Siddiqui's book and "developing credit scorecards using credit scoring for sas"
11-24-2017 09:16 AM - edited 11-24-2017 09:20 AM
Your bad percent is too small. I think your Logistic model would suffer Overdisperse Problem.
Why not using Oversample like good:bad= 1:1 or 2:1 .
11-25-2017 04:38 AM
Sorry . I never use EM. but the following could oversample.
proc surveyselect data=have sampsize=(1000 1000) out=data_oversample;
Note: assuming you have 1000 bad obs.