Building models with SAS Enterprise Miner, SAS Factory Miner, SAS Visual Data Mining and Machine Learning or just with programming

Correlated variables in classifiers

Reply
Contributor
Posts: 47

Correlated variables in classifiers

Hi,

I was wondering if someone can help me? I am using some classifiers and my dataset contains some variables with no correlation and some with high correlation. 

 

I am looking to know that how important classifiers like SVM, Neural Network, Decision Tree, Random Forest and Logistic Regression deal with highly correlated variables?

 

Do they automatically reject them for classification or we have to manage ourself?

If we have to manage ourself then how can I manage that ?

 

In all of these classifiers how the selection and rejection of features takes place?

 

 

Regards

 

SAS Employee
Posts: 121

Re: Correlated variables in classifiers

[ Edited ]

Correlation among input variables could be a very important issue in classical regression where the structure of the model was critical to generating useful results and interpretation.  In most data mining scenarios, you have far more data than was available to historical approaches as well as powerful methods (linear & nonlinear) that allow you to model relationships using flexible models which adapt to your data.  You can use holdout data to empirically validate the relationships with data rather than relying on assumptions.  The amount of interpretation available differs from model to model.   Trees provide simple interpretability while neural network and SVM models do not lend themselves to interpretation.  Correlation is a concern for interpretation of simple regression models but interpretation is not meaningful if the model is inadequate which they often are.  

 

If you would benefit from broader training in using these methods, check out the training available at 

 

  http://support.sas.com/training/us/paths/dm.html

 

where you can get a better understand about how these different models can be used. 

 

Hope this helps!

Doug

 

 

Ask a Question
Discussion stats
  • 1 reply
  • 176 views
  • 1 like
  • 2 in conversation