Building models with SAS Enterprise Miner, SAS Factory Miner, SAS Visual Data Mining and Machine Learning or just with programming

Why is lift for neural network that stable in SAS Viya demo?

Accepted Solution Solved
Reply
New Contributor
Posts: 4
Accepted Solution

Why is lift for neural network that stable in SAS Viya demo?

I'm looking at the SAS Viya machine learing demo. It races some machine Learning algorithms against each other on a given dataset. All models produce almost equally good "lift" as shown in lift diagrams in the output.

 

If you tweak the Learning to perform on a smaller subset of the data; only 0.002% of the total data set (proc partition data=&casdata partition samppct=0.002Smiley Wink, most algorithms get into problems producing lift.

 

But the neural network is still performing very well. Feature or bug? I could imagine that the script does not re-initilize the network, but it is hard to guess from the calls alone.


Accepted Solutions
Solution
‎02-20-2017 08:54 AM
SAS Employee
Posts: 23

Re: Why is lift for neural network that stable in SAS Viya demo?

Mats - the short answer without running some studies of my own is that neural networks are highly adaptive and can train very accurate models with far fewer observations than many other techniques.  The tree-based models are going to be quite unstable with very few observations.  In this case you sampled all the way down to around 20 observations...even that might be sufficient for a neural network if the space it not overly nonlinear.

 

As for your last comment - it seems you are referring to what is known as warm start, where a previously trained model can be used as a starting point and refined by providing new observations.  That is NOT what is happening here, as that capability is only coming available in our upcoming release which is just over a month away.

 

Thanks for trying out the software and experimenting...please continue to provide feedback and ask questions over this forum.

 

Brett

View solution in original post


All Replies
Solution
‎02-20-2017 08:54 AM
SAS Employee
Posts: 23

Re: Why is lift for neural network that stable in SAS Viya demo?

Mats - the short answer without running some studies of my own is that neural networks are highly adaptive and can train very accurate models with far fewer observations than many other techniques.  The tree-based models are going to be quite unstable with very few observations.  In this case you sampled all the way down to around 20 observations...even that might be sufficient for a neural network if the space it not overly nonlinear.

 

As for your last comment - it seems you are referring to what is known as warm start, where a previously trained model can be used as a starting point and refined by providing new observations.  That is NOT what is happening here, as that capability is only coming available in our upcoming release which is just over a month away.

 

Thanks for trying out the software and experimenting...please continue to provide feedback and ask questions over this forum.

 

Brett

New Contributor
Posts: 4

Re: Why is lift for neural network that stable in SAS Viya demo?

Thanks a lot for the fast and clear answer. I also posted my quesion on Stack Overflow. Is it OK if I post your answer there as well? /Mats

New Contributor
Posts: 4

Re: Why is lift for neural network that stable in SAS Viya demo?

Thanks a lot for the fast and clear answer! I also posted my quesion on Stack Overflow. Is it OK if I post your answer there as well!? /Mats
SAS Employee
Posts: 23

Re: Why is lift for neural network that stable in SAS Viya demo?

Hi Mats - Certainly feel free to post my answer on Stack Overflow.  If you don't mind, would you provide the link to that posting here so that people can jump over and see what others might have to say.  Cross-posting is always good :-).

 

Thanks.

New Contributor
Posts: 4

Re: Why is lift for neural network that stable in SAS Viya demo?

Thanks again, here is my cross-posting over att Stack Overflow!

SAS Employee
Posts: 2

Re: Why is lift for neural network that stable in SAS Viya demo?

Mats, PROC NNET initializes weight random, if you specify a seed in the train statement, the initial weights are repeatable. NNET training is powered by a sophiscated nonlinear optimization solver, if the log shows "converged" status, it means the model is fit very well. 

☑ This topic is SOLVED.

Need further help from the community? Please ask a new question.

Discussion stats
  • 6 replies
  • 343 views
  • 4 likes
  • 3 in conversation