Weight Decay vs Early Stopping

[pvareschi]

Re: Neural Network Modelling

Can Weight Decay be used alongside Early Stopping, in a way to complement each other, or are the two methods mutually exclusive (page 3.8-3.9 of course text)? The demonstrationts in the course text seem to give preference to Early Stopping.

[zhongxiuliu · Accepted solution]

Both weight decay (regularization, adding weight to cost function to avoid overfitting) and early stopping (stop when performed the best at validation) are generally concept in machine learning models, not unique to neural networks. 

Many use both, and there's no clear preference which also works better than the other.  Neural network is very easy to overfit the data, so i would recommend using both