Hi Wendy, This might be a big ask but say I had three variables (Post Code, Region, income_group) for which I wanted to consolidate their levels using a decision tree. An approach will be using three decision tree nodes for the three variables. However, is it possible to use a single decision tree node, and then maybe change certain properties to allow the tree node to produce three separate new variables for the collapsed levels of those three variables?   This ... View more

The code I provided is a start to writing a complete extension node for SAS Enterprise Miner. You can modify the metadata to make the new variable an Input variable. The program actually creates score code, it is the code stored in the location identified by the EM_FLOW_EMFLOWSCORECODE macro variable. You can learn more about extension nodes by downloading "SAS Enterprise Miner Extension Nodes: Developer's Guide." Search on "SAS Extension Nodes Developer's Guide." The latest path seems to be   https://support.sas.com/documentation/cdl/en/emxndg/67980/PDF/default/emxndg.pdf   You can also learn more about extension nodes by taking the course "Extending SAS Enterprise Miner with User-Written Nodes." The documentation or the course will teach you about flow score code and publish score code.   If you consolidate levels using a Decision Tree node as described in the EM course, you change the leaf property to input, then the variable _NODE_ is exported as an input for subsequent nodes.   If done correctly, you do not need a Replacement node.   Note that the input variable created using my code is automatically given the name of the original variable with suffix "_clus" added. You can obviously change the code to do anything you like.   Regards, Terry ... View more

This post about standardization might be helpful: https://communities.sas.com/t5/SAS-Communities-Library/To-standardize-data-or-not-to-standardize-data-that-is-the/ta-p/361726 ... View more

Very thorough reply by Neville who implemented all of these methods in SAS.  A couple of supporting comments:     1. Outliers The base model used in RF is a large decision tree. Decision trees are robust to outliers, because they isolate them in small regions of the feature space. Then, since the prediction for each leaf is the average (for regression) or the majority class (for classification), being isolated in separate leaves, outliers won't influence the rest of the predictions (in the case of regression for instance, they would not impact the mean of the other leaves) 2, Validation Data -- Yes please use in teh common case of rare events where the OOB might not be sufficient.  3. Transforms - consider continous Y as with many algorithms.  ... View more

Hi Wendy, Thanks for the answer and article.. Paul ... View more

Hi Wendy, Thanks for the papers. They all look like easy reads. Thanks ... View more