BookmarkSubscribeRSS Feed
asimina
Calcite | Level 5

Hello and tahnks in Advance.

 

I would to know what is happening when i have 2 targets in my metatada and i run  Neural Network.

Is it run them as different models or its a multiTarget network where tries to fullfill 2 targets at same time?

 

For example i have T_cross and T_upsells as Targets and I want a model with one Target , based on better performance. Whars the architecure if i have more than one?

7 REPLIES 7
WendyCzika
SAS Employee

It is fitting the 2 targets simulataneously, in a single model.

asimina
Calcite | Level 5
Thanks Wendy.
I realize now that my question is not complete.
I have a binary target inside the system in 2 forms. Binary and Interval. Trying to stabilize my model. I forgot to reject the interval form from metadata. So the model that was created is based on both of them.
My issue is. How correct and how far is that model from the appropriate one?

thanks again
Stewartli
Fluorite | Level 6

Hi Wendy, 

2 or more targets in a neural network are correct. I have regression models for 7 targets respectively and want to compare them with the neural network. When model comparison was used, the neural network was only compared to one of the regression models, instead of all 7 regression models. Any advice? Thank you very much. 

Stewart

 

DougWielenga
SAS Employee

The Model Comparison node is for comparing one target variable across multiple models.  Most modeling nodes do not model multiple targets at the same time.  The Model Comparison node will produce different types of output depending on the type of target variable and it must choose a single target in order to generate results. 

 

If I understand correctly, you modeled one of each of seven different targets in seven different Regression nodes and modeled all seven targets in the Neural Network node.  In this situation, each of your Regression nodes will produce predicted outcome variable(s) for one of the seven targets while the Neural Network node produces all seven outcome variables.   

 

The easiest approach would be to simply add a Neural Network node to each of the Regression node flows so that the appropriate Neural Network model can be compared with the corresponding Regression model.  It would be more work to import the Neural Network model containing all seven sets of predictions into each of the flows and then set up the metadata to use the corresponding prediction information so that you are comparing two models built against the same target in the Model Comparison node. 

 

If you just want to score the data using the one Neural Network model which generates seven sets of predictions and the seven Regression models (each of which generate one set of predictions) and then compare them using your own coding, that is another approach.  Given you are already building separate flows for your Regression models, it likely will be far easier to just add a parallel path involving a Neural Network node in each path. Keeping the individual models separate allows you to rebuild a Neural Network model for one target with impacting any of the other Neural Network models.  

 

Hope this helps!

Doug

 

 

Stewartli
Fluorite | Level 6

Hi Doug, 

Thank you very much for your help. 

I have three questions and want to seek your advice. 

1. Is that possible to see Adjusted R-square of Neural networks in SAS Enterprise Miner?

2. Neural networks (MLP, Back-propagation) underperformed compared to regression models. I did adjust hyperparameters a couple of times. Are there any other ways to improve my NN performance?

3. I have used 7 regression models and 7 NN for 7 targets. Which NN (NN with 7 targets or NN with only 1 target) should be used when I try to score. I guess my question is that would same results be obtained for both types of NN?

Thank you. Appreciate your time.

Stewart 

DougWielenga
SAS Employee

Stewartli,

 

I'll try to respond below each of your questions:

 

  1. Is that possible to see Adjusted R-square of Neural networks in SAS Enterprise Miner?  

      Response – This statistic is not calculated.  The adjusted R-square for a regression model is used to try and avoid overfitting by adding a penalty for an increase in the number of parameters caused by adding input variables/parameters.  A Neural Networks uses every variable in every iteration in multiple ways so there is not the same notion of adding and removing variables.  As a result, it does not necessarily make sense to compute this metric for Neural Networks.  Should you find yourself with insufficient data to evaluate your models using holdout data, you probably should not be using a Neural Network to start with. 

 

  1. Neural networks (MLP, Back-propagation) underperformed compared to regression models. I did adjust hyperparameters a couple of times. Are there any other ways to improve my NN performance?

      Response – Performance in a given situation can be both data specific and metric specific.  A model that underperforms in one data set using a certain metric won't necessarily always do so with other metrics and/or with other data sets.  How to improve the fit in a given situation depends on the data itself as well as the metric you are using.  Did you stratify your target between training and validation?  Did you overfit or underfit?  Have you used enough hidden units?  Did you use too many?  Did you impute missing values? If so, how?  How is the input data distributed across the input dimensions?  What metric are you using and are you sure it is measuring something relevant to your business objective?  There simply are no general answers that always work.

 

  1. I have used 7 regression models and 7 NN for 7 targets. Which NN (NN with 7 targets or NN with only 1 target) should be used when I try to score. I guess my question is that would same results be obtained for both types of NN?

          Response – You have 7 targets so you need to generate predictions for all 7. Either way you fit the model (e.g. filtering out subsets or fitting them simultaneously), you should be able to generate scores for each of the target variables.  Even if you fit the same Neural Network, the optimization process will not necessarily iterate to the same answer due to variability in the starting values and the change to those values in subsequent iterations.  Therefore, it is unlikely they will be exactly the same but they will likely both be useful.

 

Hope this helps!

Doug

Stewartli
Fluorite | Level 6

Hi Doug, 

Thank you so much for your reply. It is very helpful. Appreciate it.

I am going to revisit the process and make sure they are acceptable. 

 

Cheers.

Stewart

sas-innovate-2024.png

Join us for SAS Innovate April 16-19 at the Aria in Las Vegas. Bring the team and save big with our group pricing for a limited time only.

Pre-conference courses and tutorials are filling up fast and are always a sellout. Register today to reserve your seat.

 

Register now!

How to choose a machine learning algorithm

Use this tutorial as a handy guide to weigh the pros and cons of these commonly used machine learning algorithms.

Find more tutorials on the SAS Users YouTube channel.

Discussion stats
  • 7 replies
  • 1669 views
  • 1 like
  • 4 in conversation