Building models with SAS Enterprise Miner, SAS Factory Miner, SAS Visual Data Mining and Machine Learning or just with programming

Variable Ranking in Random Forest

Accepted Solution Solved
Reply
User
Posts: 1
Accepted Solution

Variable Ranking in Random Forest

Hi,

I have a dataset of 957 predictors and a binary target. Many predictors are highly correlated so I want to run a random forest model to select a subset of predictors for further modeling. I set the 'Variable Selection' option of the HP Forest node to 'Yes' and then attached another HP Forest node to it so I can use only the selected variables for modeling.

 

 Capture.PNG

However, I am not sure how the variables are selected. First, when I look at the 'Variable Importance' tab of the 1st HP Forest node, the variables can be ranked by their 'Number of Splitting Rules' and 'Gini Reduction'. As I understand, 0 on both statistics mean that the variable is not important. Does it also mean Random Forest doesn't use those variables at all (as no splitting rules are made based on those variables)?  

Second, when I check the 'Variable Importance' tab of the 2nd HP Forest node, I see there are only 586 variables, but many variables are those that have '0' importance (Based on Num of Splitting Rules and Gini Reduction) based on the 'Variable Importance' ranking in the 1st HP Forest node. Can someone explain to me how does HP Forest do variable selection? 


Accepted Solutions
Solution
‎09-29-2017 07:14 AM
SAS Super FREQ
Posts: 306

Re: Variable Ranking in Random Forest

[ Edited ]

This is documented in detail in the SAS Enterprise MIner 14.1 High-Performance Procedures documentation available from the Secure documentation link at: http://go.documentation.sas.com/?docsetId=emhpprcref&docsetTarget=emhpprcref_hpforest_details28.htm&... (see the information in the paragraph above the link for information on obtaining access to this secure site).  There is a whole section under Details in the HPFOREST chapter on measuring variable importance.  Hope that helps!

Wendy

 

View solution in original post


All Replies
Solution
‎09-29-2017 07:14 AM
SAS Super FREQ
Posts: 306

Re: Variable Ranking in Random Forest

[ Edited ]

This is documented in detail in the SAS Enterprise MIner 14.1 High-Performance Procedures documentation available from the Secure documentation link at: http://go.documentation.sas.com/?docsetId=emhpprcref&docsetTarget=emhpprcref_hpforest_details28.htm&... (see the information in the paragraph above the link for information on obtaining access to this secure site).  There is a whole section under Details in the HPFOREST chapter on measuring variable importance.  Hope that helps!

Wendy

 

Occasional Contributor
Posts: 5

Re: Variable Ranking in Random Forest

Posted in reply to WendyCzika

Dear Wendy,

I can't access your provided link  http://supportprod.unx.sas.com/documentation/onlinedoc/miner/index.html , is there any other link that can explain the best practices on using Random Forest for variable selection? Thanks.

 

Nelson

SAS Super FREQ
Posts: 306

Re: Variable Ranking in Random Forest

☑ This topic is solved.

Need further help from the community? Please ask a new question.

Discussion stats
  • 3 replies
  • 693 views
  • 1 like
  • 3 in conversation