BookmarkSubscribeRSS Feed
🔒 This topic is solved and locked. Need further help from the community? Please sign in and ask a new question.
YuanNiu
Calcite | Level 5

Hi,

I have a dataset of 957 predictors and a binary target. Many predictors are highly correlated so I want to run a random forest model to select a subset of predictors for further modeling. I set the 'Variable Selection' option of the HP Forest node to 'Yes' and then attached another HP Forest node to it so I can use only the selected variables for modeling.

 

 Capture.PNG

However, I am not sure how the variables are selected. First, when I look at the 'Variable Importance' tab of the 1st HP Forest node, the variables can be ranked by their 'Number of Splitting Rules' and 'Gini Reduction'. As I understand, 0 on both statistics mean that the variable is not important. Does it also mean Random Forest doesn't use those variables at all (as no splitting rules are made based on those variables)?  

Second, when I check the 'Variable Importance' tab of the 2nd HP Forest node, I see there are only 586 variables, but many variables are those that have '0' importance (Based on Num of Splitting Rules and Gini Reduction) based on the 'Variable Importance' ranking in the 1st HP Forest node. Can someone explain to me how does HP Forest do variable selection? 

1 ACCEPTED SOLUTION

Accepted Solutions
WendyCzika
SAS Employee

This is documented in detail in the SAS Enterprise MIner 14.1 High-Performance Procedures documentation available from the Secure documentation link at: http://go.documentation.sas.com/?docsetId=emhpprcref&docsetTarget=emhpprcref_hpforest_details28.htm&... (see the information in the paragraph above the link for information on obtaining access to this secure site).  There is a whole section under Details in the HPFOREST chapter on measuring variable importance.  Hope that helps!

Wendy

 

View solution in original post

3 REPLIES 3
WendyCzika
SAS Employee

This is documented in detail in the SAS Enterprise MIner 14.1 High-Performance Procedures documentation available from the Secure documentation link at: http://go.documentation.sas.com/?docsetId=emhpprcref&docsetTarget=emhpprcref_hpforest_details28.htm&... (see the information in the paragraph above the link for information on obtaining access to this secure site).  There is a whole section under Details in the HPFOREST chapter on measuring variable importance.  Hope that helps!

Wendy

 

nelson_lee
Fluorite | Level 6

Dear Wendy,

I can't access your provided link  http://supportprod.unx.sas.com/documentation/onlinedoc/miner/index.html , is there any other link that can explain the best practices on using Random Forest for variable selection? Thanks.

 

Nelson

sas-innovate-2024.png

Join us for SAS Innovate April 16-19 at the Aria in Las Vegas. Bring the team and save big with our group pricing for a limited time only.

Pre-conference courses and tutorials are filling up fast and are always a sellout. Register today to reserve your seat.

 

Register now!

How to choose a machine learning algorithm

Use this tutorial as a handy guide to weigh the pros and cons of these commonly used machine learning algorithms.

Find more tutorials on the SAS Users YouTube channel.

Discussion stats
  • 3 replies
  • 3273 views
  • 1 like
  • 3 in conversation