https://communities.sas.com/t5/SAS-Academy-for-Data-Science/Choosing-optimisation-algorithm-for-estimation-of-Neural/m-p/651988#M842 <P>Re: Neural Network Modelling</P> <P>Are there any guidelines/suggestions on how to choose which optimisation method to use in a given situation, beside the number of weights to be estimated? (see page 3.42-3.45 of course text).</P> <P>In particular, are backprop, Qprop and Rprop ever used? And what about Double-Dogleg?<BR />In addition, page 3.43 states that "Quasi-Newton is among the fastest and most reliable algorithms for unconstrained optimisation of smooth objective functions": when would that be the case when fitting a Neural Network, given that the error function tend to be highly non-linear?</P> Sat, 30 May 2020 16:41:20 GMT pvareschi 2020-05-30T16:41:20Z https://communities.sas.com/t5/SAS-Academy-for-Data-Science/Choosing-optimisation-algorithm-for-estimation-of-Neural/m-p/651988#M842 <P>Re: Neural Network Modelling</P> <P>Are there any guidelines/suggestions on how to choose which optimisation method to use in a given situation, beside the number of weights to be estimated? (see page 3.42-3.45 of course text).</P> <P>In particular, are backprop, Qprop and Rprop ever used? And what about Double-Dogleg?<BR />In addition, page 3.43 states that "Quasi-Newton is among the fastest and most reliable algorithms for unconstrained optimisation of smooth objective functions": when would that be the case when fitting a Neural Network, given that the error function tend to be highly non-linear?</P> Sat, 30 May 2020 16:41:20 GMT https://communities.sas.com/t5/SAS-Academy-for-Data-Science/Choosing-optimisation-algorithm-for-estimation-of-Neural/m-p/651988#M842 pvareschi 2020-05-30T16:41:20Z