Correlated variables in classifiers

geniusgenie — Fri, 02 Jun 2017 22:09:12 GMT

Hi,

I was wondering if someone can help me? I am using some classifiers and my dataset contains some variables with no correlation and some with high correlation.

I am looking to know that how important classifiers like SVM, Neural Network, Decision Tree, Random Forest and Logistic Regression deal with highly correlated variables?

Do they automatically reject them for classification or we have to manage ourself?

If we have to manage ourself then how can I manage that ?

In all of these classifiers how the selection and rejection of features takes place?

Regards

Re: Correlated variables in classifiers

DougWielenga — Mon, 07 Aug 2017 17:44:01 GMT

Correlation among input variables could be a very important issue in classical regression where the structure of the model was critical to generating useful results and interpretation. In most data mining scenarios, you have far more data than was available to historical approaches as well as powerful methods (linear & nonlinear) that allow you to model relationships using flexible models which adapt to your data. You can use holdout data to empirically validate the relationships with data rather than relying on assumptions. The amount of interpretation available differs from model to model. Trees provide simple interpretability while neural network and SVM models do not lend themselves to interpretation. Correlation is a concern for interpretation of simple regression models but interpretation is not meaningful if the model is inadequate which they often are.

If you would benefit from broader training in using these methods, check out the training available at

http://support.sas.com/training/us/paths/dm.html

where you can get a better understand about how these different models can be used.

Hope this helps!

Doug

topic Re: Correlated variables in classifiers in SAS Data Science

Correlated variables in classifiers

Re: Correlated variables in classifiers