topic Re: Why is lift for neural network that stable in SAS Viya demo? in SAS Data Science

Why is lift for neural network that stable in SAS Viya demo?

Mats — Thu, 16 Feb 2017 10:45:37 GMT

I'm looking at the SAS Viya machine learing demo. It races some machine Learning algorithms against each other on a given dataset. All models produce almost equally good "lift" as shown in lift diagrams in the output.

If you tweak the Learning to perform on a smaller subset of the data; only 0.002% of the total data set (proc partition data=&casdata partition samppct=0.002;), most algorithms get into problems producing lift.

But the neural network is still performing very well. Feature or bug? I could imagine that the script does not re-initilize the network, but it is hard to guess from the calls alone.

Re: Why is lift for neural network that stable in SAS Viya demo?

BrettWujek — Fri, 17 Feb 2017 19:56:06 GMT

Mats - the short answer without running some studies of my own is that neural networks are highly adaptive and can train very accurate models with far fewer observations than many other techniques. The tree-based models are going to be quite unstable with very few observations. In this case you sampled all the way down to around 20 observations...even that might be sufficient for a neural network if the space it not overly nonlinear.

As for your last comment - it seems you are referring to what is known as warm start, where a previously trained model can be used as a starting point and refined by providing new observations. That is NOT what is happening here, as that capability is only coming available in our upcoming release which is just over a month away.

Thanks for trying out the software and experimenting...please continue to provide feedback and ask questions over this forum.

Brett

Re: Why is lift for neural network that stable in SAS Viya demo?

Xinmin — Fri, 17 Feb 2017 20:56:53 GMT

Mats, PROC NNET initializes weight random, if you specify a seed in the train statement, the initial weights are repeatable. NNET training is powered by a sophiscated nonlinear optimization solver, if the log shows "converged" status, it means the model is fit very well.

Re: Why is lift for neural network that stable in SAS Viya demo?

Mats — Sat, 18 Feb 2017 15:39:23 GMT

Thanks a lot for the fast and clear answer. I also posted my quesion on Stack Overflow. Is it OK if I post your answer there as well? /Mats

Re: Why is lift for neural network that stable in SAS Viya demo?

Mats — Mon, 20 Feb 2017 13:55:30 GMT

Thanks a lot for the fast and clear answer! I also posted my quesion on Stack Overflow. Is it OK if I post your answer there as well!? /Mats

Re: Why is lift for neural network that stable in SAS Viya demo?

BrettWujek — Mon, 20 Feb 2017 14:00:15 GMT

Hi Mats - Certainly feel free to post my answer on Stack Overflow. If you don't mind, would you provide the link to that posting here so that people can jump over and see what others might have to say. Cross-posting is always good :-).

Thanks.

Re: Why is lift for neural network that stable in SAS Viya demo?

Mats — Mon, 20 Feb 2017 14:09:49 GMT

Thanks again, here is my cross-posting over att Stack Overflow!