Neural Network: Auto-tuning in SAS Visual Data Mining and Machine Learning

[Stewartli]

Hi guys,

My dataset contains 113 variables. The half of it was selected to develop the neural network. It does not make sense if you look at the graph (but only one hidden layer was included). Thereafter, auto-tuning (SAS VDMML) was used, the neural network became a little crazy (4 hidden layers were included). Can anybody tell me what is wrong with it? Thank you very much.

Regards,

Stewart

[DougWielenga · Accepted solution]

It is not clear that anything is wrong.   A single hidden layer neural  network is a universal approximator which means that a single hidden layer neural network can model any surface within a specified amount of error given a sufficient number hidden units.  Moving to multiple hidden layers won't necessarily improve the model but just changes the architecture of the network so that there might be fewer hidden units needed at each layer.  Neither is guaranteed to produce a better-fitting model.  You do need to be careful to avoid overfitting but you can overfit a single hidden layer network as easily as a network with more hidden layers.  

Hope this helps!
Doug

[Stewartli]

Hi Doug,

Thank you very much. It does help. 

Stewart

[BrettWujek]

As Doug states, it's not clear that there is necessarily anything wrong here.  But yeah, the final model from the autotuning process is definitely more (overly) complex. I strongly suggest that you try autotuning your model with code in SAS Studio so that you can see much more detail and have control over the ranges of the hyperparameters (e.g., if you just wanted a single hidden layer you can tell it to not autotune that), and you can see the top 10 models from the process so that you can perform tradeoffs.

For an example of using the SAS Studio Neural Network task to autotune a neural network, see this video. The first part is just all about the process...at the 5:25 mark I start showing how to do it with the task.

[Stewartli]

Hi Brett, 

Thank you very much for that. 

Yes, the video did inspire me to consider auto-tuning in my research in the first place. 

Regards, 

Stewart