Autotuning in E miner

[jmehul]

Hi All,
I am using SAS E miner 14.3. There are some predefined parameters for any model like depth = 6 Max Branch=2 etc.

Is there any way in which I can perform the grid search to find the best parameters.
I want to know how can I autotune the parameters for the model on the basis of cross-validation error or any other method.

[DougWielenga · Accepted solution]

Is there any way in which I can perform the grid search to find the best parameters.
I want to know how can I autotune the parameters for the model on the basis of cross-validation error or any other method.

It sounds like you are talking about a Decision Tree model.   When building this type of model, interpretation is often as important as overall performance.   There are likely to be many different trees that perform very similarly on hold-out data and the best model for your business question will depend on what question you are trying to ask. 

For example, if you are trying to identify different groups of people to market to, you might set a high minimum number of observations for a terminal leaf since you don't want to create different marketing strategies for a large number of tiny groups of customers.   If you are looking for Fraud, however, you might be interested in growing the tree deeper to find unusual terminal nodes with a small number of entries.   Even if you end up comparing different types of models (not just trees), you might still choose a simpler model that has some interpretation rather than fitting a complex model that has none.   In the case of a tree, though, the interpretation is still typically of great interest.  If interpretation is not the goal, a random forest (which is comprised of many trees) is likely to provide a higher assessment value than a single tree model.  

While it is easy to build out several different trees in parallel to see how different settings impact the building of the tree, there is no way for the software to anticipate which tree will provide the appropriate balance of interpretation and performance so no 'grid search' is available. 

Hope this helps!

Doug