topic Re: Gini OOT VS IN SAMPLE in SAS Data Science

Gini OOT VS IN SAMPLE

Ronein — Tue, 31 Dec 2024 10:18:34 GMT

Hello
I am building a credit score model
What is acceptable difference in Gini between in-sample( train data) to out of time ? For example; Gini in in-sample is 80% and Gini in out of time data is 82%. Is it good or bad? I afraid that 2% difference means model is not good?

Re: Gini OOT VS IN SAMPLE

Ksharp — Wed, 01 Jan 2025 01:59:20 GMT

Do you want to perform Superiority Testing or Noninferiority Testing ?
But that concept is not from Credit Score, is from bistatistic .

https://communities.sas.com/t5/Statistical-Procedures/How-to-run-Power-analysis-to-capture-sales-lift/m-p/946391/highlight/false#M47288

http://support.sas.com/kb/48/616.html
https://support.sas.com/kb/50/700.html

Re: Gini OOT VS IN SAMPLE

sbxkoenk — Tue, 07 Jan 2025 13:09:55 GMT

@Ronein wrote:
Hello
I am building a credit score model
What is acceptable difference in Gini between in-sample( train data) to out of time ? For example; Gini in in-sample is 80% and Gini in out of time data is 82%. Is it good or bad? I afraid that 2% difference means model is not good?

Gini in-sample (training data) = 80%

Gini out-of-sample and out-of-time = 82%

The Gini coefficient measures separation power. Similar to how it is used in economics to measure inequality. However for credit risk, the higher the Gini, the better.

So, I don't see the problem ... unless you fear a "too good to be true"-type of error.

Ciao, Koen