BookmarkSubscribeRSS Feed
Ronein
Onyx | Level 15
Hello
Let's say that someone else developed a model that the response varaible is binary indicator of failure ( default) and the predictor varaibles are credit card usage behavior. I don't know exactly how the model was built and what was checked regarding credit card usage but in the bottom line for each customer I received calculated pd ( probability of default).
I received pd for customers in different times.
I was asked to check the model in different data ( data of customers from different times).
I thought to do following-
Calculate Gini coefficient for each out of time data .
Calculate 20% capture rate for each out of time data.
My question- what other checks do you recommend me to do that test the model and its stability?
2 REPLIES 2
Ksharp
Super User
"I was asked to check the model in different data"
Check for what ? Check/Predict pd of customer at different times ? It would lead you to build Mixed Model for repeated measure. Check PROC GLIMMIX .

"what other checks do you recommend me to do that test the model and its stability?"
If your 'stability' means goodness of fitness test of model, then check GOF option of MODEL statement of PROC LOGISTIC.

If you are building a Credit Score Card , you could check PSI index to see whether the population of customer is change or not .
https://communities.sas.com/t5/SAS-Programming/PSI-stability-index/m-p/802144
https://communities.sas.com/t5/SAS-Programming/Is-my-PSI-Population-Stability-Index-calculation-code...

Check @Rick_SAS blogs:
https://blogs.sas.com/content/iml/2019/02/20/easier-calibration-plot-sas.html
https://blogs.sas.com/content/iml/2018/05/14/calibration-plots-in-sas.html
https://blogs.sas.com/content/iml/2016/11/28/goodness-of-fit-large-small-samples.html
Season
Barite | Level 11

It seems like you wish to validate a model on your data. Given that the dependent variable is binary, you can employ the concordance statistic (statistic) to validate the model.

hackathon24-white-horiz.png

2025 SAS Hackathon: There is still time!

Good news: We've extended SAS Hackathon registration until Sept. 12, so you still have time to be part of our biggest event yet – our five-year anniversary!

Register Now

How to choose a machine learning algorithm

Use this tutorial as a handy guide to weigh the pros and cons of these commonly used machine learning algorithms.

Find more tutorials on the SAS Users YouTube channel.

Discussion stats
  • 2 replies
  • 7217 views
  • 0 likes
  • 3 in conversation