Correlated variables in classifiers

[geniusgenie]

Hi,

I was wondering if someone can help me? I am using some classifiers and my dataset contains some variables with no correlation and some with high correlation. 

I am looking to know that how important classifiers like SVM, Neural Network, Decision Tree, Random Forest and Logistic Regression deal with highly correlated variables?

Do they automatically reject them for classification or we have to manage ourself?

If we have to manage ourself then how can I manage that ?

In all of these classifiers how the selection and rejection of features takes place?

Regards

[DougWielenga]

Correlation among input variables could be a very important issue in classical regression where the structure of the model was critical to generating useful results and interpretation.  In most data mining scenarios, you have far more data than was available to historical approaches as well as powerful methods (linear & nonlinear) that allow you to model relationships using flexible models which adapt to your data.  You can use holdout data to empirically validate the relationships with data rather than relying on assumptions.  The amount of interpretation available differs from model to model.   Trees provide simple interpretability while neural network and SVM models do not lend themselves to interpretation.  Correlation is a concern for interpretation of simple regression models but interpretation is not meaningful if the model is inadequate which they often are.  

If you would benefit from broader training in using these methods, check out the training available at 

  http://support.sas.com/training/us/paths/dm.html

where you can get a better understand about how these different models can be used. 

Hope this helps!

Doug