turn on suggestions

Auto-suggest helps you quickly narrow down your search results by suggesting possible matches as you type.

Showing results for

Find a Community

- Home
- /
- Analytics
- /
- Data Mining
- /
- Variable importance

Topic Options

- Subscribe to RSS Feed
- Mark Topic as New
- Mark Topic as Read
- Float this Topic for Current User
- Bookmark
- Subscribe
- Printer Friendly Page

- Mark as New
- Bookmark
- Subscribe
- Subscribe to RSS Feed
- Highlight
- Email to a Friend
- Report Inappropriate Content

04-14-2012 08:00 PM

I can't reconcile the variable importance using decision trees and regression on sas on demand for academics 6.2 miner. In particular I'm using the data PVA97NK. In otherwords, their ranking are not the same.

Any help is welcomed.

Thanks.

Accepted Solutions

Solution

Tuesday

- Mark as New
- Bookmark
- Subscribe
- Subscribe to RSS Feed
- Highlight
- Email to a Friend
- Report Inappropriate Content

Tuesday

The regression and tree-based models work in very different ways. The regression builds a model that attempts to describe the relationship between the target and involves estimating parameters that are then multiplied by variable values to predict the outcome. Decision trees do not build a parametric model and must be evaluated differently.

There are also many differences in how regression and decision tree models perform on different data sets.

* Regression models do a better job of describing smoothly changing relationships than a decision tree which would need to create many cutpoints to accomplish the same.

* Decision trees handle missing values whereas regression models must have those values imputed meaning the two methods are not even working on the same training data when missing values are present in the original training data.

* Regression models require you to specify the exact functional form of the relationship whereas decision trees do not assume any specific relationship.

* Decision trees are (typically) far more flexible than regression models allowing them to automatically model complex interactions and non-linear relationships that would need to be described explicitly in a regression model.

* You cannot pick one of these methods as the superior method overall since it depends on the data you are fitting

For all these reasons, you should not expect that the variable importance reported by a decision tree is going to parallel what you would see from the corresponding regression model fit to the same data.

I hope this helps!

Doug

All Replies

Solution

Tuesday

- Mark as New
- Bookmark
- Subscribe
- Subscribe to RSS Feed
- Highlight
- Email to a Friend
- Report Inappropriate Content

Tuesday

The regression and tree-based models work in very different ways. The regression builds a model that attempts to describe the relationship between the target and involves estimating parameters that are then multiplied by variable values to predict the outcome. Decision trees do not build a parametric model and must be evaluated differently.

There are also many differences in how regression and decision tree models perform on different data sets.

* Regression models do a better job of describing smoothly changing relationships than a decision tree which would need to create many cutpoints to accomplish the same.

* Decision trees handle missing values whereas regression models must have those values imputed meaning the two methods are not even working on the same training data when missing values are present in the original training data.

* Regression models require you to specify the exact functional form of the relationship whereas decision trees do not assume any specific relationship.

* Decision trees are (typically) far more flexible than regression models allowing them to automatically model complex interactions and non-linear relationships that would need to be described explicitly in a regression model.

* You cannot pick one of these methods as the superior method overall since it depends on the data you are fitting

For all these reasons, you should not expect that the variable importance reported by a decision tree is going to parallel what you would see from the corresponding regression model fit to the same data.

I hope this helps!

Doug