I occupy myself with credit scoring, I use Enterprise Miner.
How to prevent overfitting when training scorecard model ?
I detect this phenomena by comparing train and validation gini coefficients. Mine are 0.53 and 0.43 respectively.
In the Scorecard node, you can change the Selection Model property to anything but "None", then for the Criterion property, use either Validation Error or Validation Misclassification to help with overfitting of the training data.
In the Scorecard node, you can change the Selection Model property to anything but "None", then for the Criterion property, use either Validation Error or Validation Misclassification to help with overfitting of the training data.
now gini equals 0.47 / 0.46.
Thanks a lot!
Your advice worked pretty well but now I have to few variables. The number changed from 10 to 5.
What can I do to prevent this baheavior?
You can try adjusting the other properties in the Model Selection section to relax the criterion for including effects in the model, or you can force certain effects into the model.
Don't miss out on SAS Innovate - Register now for the FREE Livestream!
Can't make it to Vegas? No problem! Watch our general sessions LIVE or on-demand starting April 17th. Hear from SAS execs, best-selling author Adam Grant, Hot Ones host Sean Evans, top tech journalist Kara Swisher, AI expert Cassie Kozyrkov, and the mind-blowing dance crew iLuminate! Plus, get access to over 20 breakout sessions.
Use this tutorial as a handy guide to weigh the pros and cons of these commonly used machine learning algorithms.
Find more tutorials on the SAS Users YouTube channel.