Variable Ranking in Random Forest

[YuanNiu]

Hi,

I have a dataset of 957 predictors and a binary target. Many predictors are highly correlated so I want to run a random forest model to select a subset of predictors for further modeling. I set the 'Variable Selection' option of the HP Forest node to 'Yes' and then attached another HP Forest node to it so I can use only the selected variables for modeling.

 

However, I am not sure how the variables are selected. First, when I look at the 'Variable Importance' tab of the 1st HP Forest node, the variables can be ranked by their 'Number of Splitting Rules' and 'Gini Reduction'. As I understand, 0 on both statistics mean that the variable is not important. Does it also mean Random Forest doesn't use those variables at all (as no splitting rules are made based on those variables)?  

Second, when I check the 'Variable Importance' tab of the 2nd HP Forest node, I see there are only 586 variables, but many variables are those that have '0' importance (Based on Num of Splitting Rules and Gini Reduction) based on the 'Variable Importance' ranking in the 1st HP Forest node. Can someone explain to me how does HP Forest do variable selection? 

[WendyCzika · Accepted solution]

This is documented in detail in the SAS Enterprise MIner 14.1 High-Performance Procedures documentation available from the Secure documentation link at: http://go.documentation.sas.com/?docsetId=emhpprcref&docsetTarget=emhpprcref_hpforest_details28.htm&... (see the information in the paragraph above the link for information on obtaining access to this secure site).  There is a whole section under Details in the HPFOREST chapter on measuring variable importance.  Hope that helps!

Wendy

[nelson_lee]

Dear Wendy,

I can't access your provided link  http://supportprod.unx.sas.com/documentation/onlinedoc/miner/index.html , is there any other link that can explain the best practices on using Random Forest for variable selection? Thanks.

Nelson

[WendyCzika]

Try this new link:

http://documentation.sas.com/?docsetId=emhpprcref&docsetTarget=emhpprcref_hpforest_details28.htm&doc...