topic Variable Ranking in Random Forest in SAS Data Science

Variable Ranking in Random Forest

YuanNiu — Tue, 04 Oct 2016 21:13:44 GMT

Hi,

I have a dataset of 957 predictors and a binary target. Many predictors are highly correlated so I want to run a random forest model to select a subset of predictors for further modeling. I set the 'Variable Selection' option of the HP Forest node to 'Yes' and then attached another HP Forest node to it so I can use only the selected variables for modeling.

However, I am not sure how the variables are selected. First, when I look at the 'Variable Importance' tab of the 1st HP Forest node, the variables can be ranked by their 'Number of Splitting Rules' and 'Gini Reduction'. As I understand, 0 on both statistics mean that the variable is not important. Does it also mean Random Forest doesn't use those variables at all (as no splitting rules are made based on those variables)?

Second, when I check the 'Variable Importance' tab of the 2nd HP Forest node, I see there are only 586 variables, but many variables are those that have '0' importance (Based on Num of Splitting Rules and Gini Reduction) based on the 'Variable Importance' ranking in the 1st HP Forest node. Can someone explain to me how does HP Forest do variable selection?

Re: Variable Ranking in Random Forest

WendyCzika — Fri, 29 Sep 2017 11:14:32 GMT

This is documented in detail in the SAS Enterprise MIner 14.1 High-Performance Procedures documentation available from the Secure documentation link at: http://go.documentation.sas.com/?docsetId=emhpprcref&docsetTarget=emhpprcref_hpforest_details28.htm&docsetVersion=14.2&locale=en (see the information in the paragraph above the link for information on obtaining access to this secure site). There is a whole section under Details in the HPFOREST chapter on measuring variable importance. Hope that helps!

Wendy

Re: Variable Ranking in Random Forest

nelson_lee — Thu, 21 Sep 2017 07:29:31 GMT

Dear Wendy,

I can't access your provided link http://supportprod.unx.sas.com/documentation/onlinedoc/miner/index.html , is there any other link that can explain the best practices on using Random Forest for variable selection? Thanks.

Nelson

Re: Variable Ranking in Random Forest

WendyCzika — Fri, 22 Sep 2017 01:07:17 GMT

Try this new link:

http://documentation.sas.com/?docsetId=emhpprcref&docsetTarget=emhpprcref_hpforest_details28.htm&docsetVersion=14.2&locale=en