topic Re: Check pd model in SAS Data Science

Check pd model

Ronein — Sat, 01 Feb 2025 10:53:32 GMT

Hello
Let's say that someone else developed a model that the response varaible is binary indicator of failure ( default) and the predictor varaibles are credit card usage behavior. I don't know exactly how the model was built and what was checked regarding credit card usage but in the bottom line for each customer I received calculated pd ( probability of default).
I received pd for customers in different times.
I was asked to check the model in different data ( data of customers from different times).
I thought to do following-
Calculate Gini coefficient for each out of time data .
Calculate 20% capture rate for each out of time data.
My question- what other checks do you recommend me to do that test the model and its stability?

Re: Check pd model

Ksharp — Sun, 02 Feb 2025 01:33:55 GMT

"I was asked to check the model in different data"
Check for what ? Check/Predict pd of customer at different times ? It would lead you to build Mixed Model for repeated measure. Check PROC GLIMMIX .

"what other checks do you recommend me to do that test the model and its stability?"
If your 'stability' means goodness of fitness test of model, then check GOF option of MODEL statement of PROC LOGISTIC.

If you are building a Credit Score Card , you could check PSI index to see whether the population of customer is change or not .
https://communities.sas.com/t5/SAS-Programming/PSI-stability-index/m-p/802144
https://communities.sas.com/t5/SAS-Programming/Is-my-PSI-Population-Stability-Index-calculation-code-correctly/m-p/940832

Check @Rick_SAS blogs:
https://blogs.sas.com/content/iml/2019/02/20/easier-calibration-plot-sas.html
https://blogs.sas.com/content/iml/2018/05/14/calibration-plots-in-sas.html
https://blogs.sas.com/content/iml/2016/11/28/goodness-of-fit-large-small-samples.html

Re: Check pd model

Season — Thu, 24 Apr 2025 07:51:23 GMT

It seems like you wish to validate a model on your data. Given that the dependent variable is binary, you can employ the concordance statistic (c statistic) to validate the model.