Choosing optimisation algorithm for estimation of Neural Networks parameters

[pvareschi]

Re: Neural Network Modelling

Are there any guidelines/suggestions on how to choose which optimisation method to use in a given situation, beside the number of weights to be estimated? (see page 3.42-3.45 of course text).

In particular, are backprop, Qprop and Rprop ever used? And what about Double-Dogleg?
In addition, page 3.43 states that "Quasi-Newton is among the fastest and most reliable algorithms for unconstrained optimisation of smooth objective functions": when would that be the case when fitting a Neural Network, given that the error function tend to be highly non-linear?