topic Re: How to interpret Credit Scorecard results in New SAS User

How to interpret Credit Scorecard results

mackbox — Wed, 22 May 2019 12:48:28 GMT

Hi all,

I tried to fit a credit scorecard using the Enterprise Miner 14.3, and I found the results a bit confusing. As you see, the empirical odds of the training and validation data goes down and up. I expected it not to go back up again. I have tried to remove the variables one after another to see if any are causing the kink, but it seems to persist even when there are just one or two variables left. Any thoughts on what is causing this?

Many thanks

Re: How to interpret Credit Scorecard results

PaigeMiller — Wed, 22 May 2019 13:10:03 GMT

Clearly, something is wrong, the curve should be relatively monotonic decreasing, and the odds should not go up at the right side of the scale. You will have to do some digging to find out what the problem is.

Question: did you fit the model on only loans that were actually booked? This will cause bias in your model and in the odds ratios; but usually this bias shows up in the lower end of the scores, not in the higher end of the scores. There could be other bias in the higher end, although I am not able to specifically state a reason. Another problem could be a very small number of loans with a score above 300, in which case maybe your binning is wrong or needs to be modified.

Re: How to interpret Credit Scorecard results

mackbox — Wed, 22 May 2019 13:17:59 GMT

Do you mean the binning of the scorecard points or the variables in the interactive grouping step?

Re: How to interpret Credit Scorecard results

PaigeMiller — Wed, 22 May 2019 13:22:23 GMT

Probably the interactive grouping step, but as I said, you are going to have to do some digging.

And of course, there may be another explanation that I haven't thought of.

Re: How to interpret Credit Scorecard results

mackbox — Wed, 22 May 2019 13:29:18 GMT

Thanks a lot PaigeMiller. It's curious though when I pulled the numbers from the gains table and calculate the log of the bad over the goods manually, it doesn't seem to match the empirical odds on the chart at all. I wonder why?