topic Re: Check power of credit score model for good customers only (without Indeterminate customers) in SAS Data Science

Check power of credit score model for good customers only (without Indeterminate customers)

Ronein — Thu, 26 Dec 2024 01:18:50 GMT

Hello
I am building a credit score model.
I almost finished build the model and I calculated Gini .
The model is developed based on data from Jan 2023 and then check default in Feb 2023 till Jan 2024( 12 months).
Note that the data divided to train (70%) and test (30%).
I was asked to do the following-
Find customers with very bad elements that are "Indeterminate" (in Jan 2023) but still not in default (in Jan 2023).
It means that these customers are very  close to be defaulted and already have days past due (but still not enough to be in default) .
I read that these type of customers are called "Indeterminate".
My question-
The task is to find if the model has good separation power for the "good " customers (customers who are not Indeterminate).
The idea is that for customers who are not indeterminate it is more difficult to predict default.
What is the way to check it( if the modelhas good separation power for the "good " customers)?

Re: Check power of credit score model for good customers only (without Indeterminate customers)

Ksharp — Thu, 26 Dec 2024 01:16:40 GMT

What do you mean by POWER ?
This concept is from biostatistics ,NOT credit risk.
If you want see the power of distinguish of model:
1) check "confusion matrix"
2)check ROC curve:
https://support.sas.com/kb/52/973.html
https://support.sas.com/kb/45/339.html
3)check goodness-of-fit test of logistic model:
https://blogs.sas.com/content/iml/2019/02/20/easier-calibration-plot-sas.html
https://blogs.sas.com/content/iml/2018/05/14/calibration-plots-in-sas.html
https://blogs.sas.com/content/iml/2019/05/08/stability-binary-classifier.html

Re: Check power of credit score model for good customers only (without Indeterminate customers)

Ronein — Thu, 26 Dec 2024 01:21:01 GMT

There is gini that common to use. My question is about the Indeterminate customers.how should I.make the analysis?

Re: Check power of credit score model for good customers only (without Indeterminate customers)

Ksharp — Thu, 26 Dec 2024 02:11:21 GMT

gini is the same as roc:
gini =2*roc-1

You should firstly identify these " Indeterminate customers" ,then do ROC analysis or GOF of logistic ,or Confusion Matrix .

Re: Check power of credit score model for good customers only (without Indeterminate customers)

SASKiwi — Fri, 27 Dec 2024 21:17:47 GMT

What percentage of your customers are "bad" but not in default versus good? In my experience, a good lending portfolio will have over 90% good customers, less than 3% in default and the remainder with minor arrears. How does the gini of your "indeterminate" customers compare with your good customers? I would expect to see a higher gini for indeterminate's as these are the most likely to go into default.