Methodology on how to build a neural network

[pvareschi]

Re: Neural Network Modelling

I would appreciate some feedback on how a neural network should be developed in practice.

What I mean is that the course text presents several interesting topics and techniques but I feel that it does not make it clear how those elements fits together.

For instance, should variable selection be done before or after defining a network architecture in terms of hidden units required? Asking this because all three methods presented in chapter 4 (i.e. Hinton diagrams, Importance Metric and Sensitivity-Based Pruning) assume a network structure is already known...however, the optimisazion of the number of hidden units, for instance by using Sequential Network Construction, may be affected by having too many inputs...so, which one should come first?

Secondly, would it be correct to say that Early Stopping should always be tried as a starting point and other techniques could be used to refine that initial network?

[gcjfernandez · Accepted solution]

Re: Neural Network Modelling

I would appreciate some feedback on how a neural network should be developed in practice.

What I mean is that the course text presents several interesting topics and techniques but I feel that it does not make it clear how those elements fits together.

For instance, should variable selection be done before or after defining a network architecture in terms of hidden units required? Asking this because all three methods presented in chapter 4 (i.e. Hinton diagrams, Importance Metric and Sensitivity-Based Pruning) assume a network structure is already known...however, the optimisazion of the number of hidden units, for instance by using Sequential Network Construction, may be affected by having too many inputs...so, which one should come first?

My Answer:

Unlike a decision tree or logistic regression there is no built-in variable selection for pre-screening redundant variables is available in NN. Therefore, pre-screening step is a recommended best practice in NN model development. Otherwise model convergence issues might occur.

Secondly, would it be correct to say that Early Stopping should always be tried as a starting point and other techniques could be used to refine that initial network?

My Answer:

In predictive model building during the model development and in model assessment optimizing complexity of the model is always performed using validation assessment statistics. Therefore, during NN model development or during model assessment Early stopping using the validation data is always recommended. 

I am attaching a classic paper on over-fitting for your reference: