topic Re: scorecard overfitting in SAS Data Science

scorecard overfitting

ManOfHonor — Mon, 27 Nov 2017 13:27:54 GMT

I occupy myself with credit scoring, I use Enterprise Miner.

How to prevent overfitting when training scorecard model ?

I detect this phenomena by comparing train and validation gini coefficients. Mine are 0.53 and 0.43 respectively.

Re: scorecard overfitting

WendyCzika — Tue, 28 Nov 2017 14:09:37 GMT

In the Scorecard node, you can change the Selection Model property to anything but "None", then for the Criterion property, use either Validation Error or Validation Misclassification to help with overfitting of the training data.

Re: scorecard overfitting

ManOfHonor — Wed, 29 Nov 2017 07:27:43 GMT

now gini equals 0.47 / 0.46.

Thanks a lot!

Re: scorecard overfitting

ManOfHonor — Wed, 29 Nov 2017 10:54:26 GMT

Your advice worked pretty well but now I have to few variables. The number changed from 10 to 5.

What can I do to prevent this baheavior?

Re: scorecard overfitting

WendyCzika — Wed, 29 Nov 2017 13:59:05 GMT

You can try adjusting the other properties in the Model Selection section to relax the criterion for including effects in the model, or you can force certain effects into the model.