Gini OOT VS IN SAMPLE

[Ronein]

Hello
I am building a credit score model
What is acceptable difference in Gini between in-sample( train data) to out of time ? For example; Gini in in-sample is 80% and Gini in out of time data is 82%. Is it good or bad? I afraid that 2% difference means model is not good?

[Ksharp]

Do you want to perform Superiority Testing or Noninferiority Testing ?
But that concept is not from Credit Score, is from bistatistic .

https://communities.sas.com/t5/Statistical-Procedures/How-to-run-Power-analysis-to-capture-sales-lif...

http://support.sas.com/kb/48/616.html
https://support.sas.com/kb/50/700.html

[sbxkoenk]

---

@Ronein wrote:
Hello
I am building a credit score model
What is acceptable difference in Gini between in-sample( train data) to out of time ? For example; Gini in in-sample is 80% and Gini in out of time data is 82%. Is it good or bad? I afraid that 2% difference means model is not good?

---

Gini in-sample (training data) = 80%

Gini out-of-sample and out-of-time = 82%

The Gini coefficient measures separation power. Similar to how it is used in economics to measure inequality. However for credit risk, the higher the Gini, the better.

So, I don't see the problem ... unless you fear a "too good to be true"-type of error.

Ciao, Koen