Model honest assessment

[ajkalale]

Is the test data and the validation dataset the same?

[Reeza · Accepted solution]

The terms can vary a bit. But the idea is three datasets:

1. training - used to develop a model
2. validation - used to calibrate model. If model doesn’t perform well, revisit model and use this data to validate
3. test - used in final stages, to get final model accuracy  numbers. Used to prevent over fitting and provide an 'honest assessment’. You would not retrain a model if this didn’t score well