I am using SAS EM to grow a Decision Tree. I was asked by a reviewer if a "minimum difference" could be set at the terminal nodes. That is, could a threshold be set such that a parent node will not split unless the difference in the % event reaches a critical threshold value across nodes?
For example: Prune back to the parent node, any terminal nodes where the % difference is less than 5%.
The question was raised in the context of "clinically-meaningful" differences across terminal nodes vs. "statistically-meaningful" differences.
Is this minimal difference something I could manually set in SAS-EM?
Thank you!
Josh
No: The procedure that underlies the EM decision tree node does not have an option to prune close siblings. One would have to resort to advanced SAS data step programming, outputing the nodes statistics, searching for leaves that are too close, and pruning their parents. Some key elements might be:
PROC ARBOR INMODEL = savedTreeDataTable;
SAVE STATSBYNODE= statsDataTable TOPOLOGY=topologyDataTable.
Write data step code to use the topologyDataTable to add a ParentNode variable in the statsDataTable, search the statsDataTable for siblings that are too close, and outputting a macro variable containing a list of the ParentNodes of the close siblings. Then:
PROC ARBOR INMODEL=savedTreeDataTable;
PRUNE &listOfNodes;
SAVE model=prunedTreeDataTable;
This is the simplist way I can think of doing it.
-Padraic
No: The procedure that underlies the EM decision tree node does not have an option to prune close siblings. One would have to resort to advanced SAS data step programming, outputing the nodes statistics, searching for leaves that are too close, and pruning their parents. Some key elements might be:
PROC ARBOR INMODEL = savedTreeDataTable;
SAVE STATSBYNODE= statsDataTable TOPOLOGY=topologyDataTable.
Write data step code to use the topologyDataTable to add a ParentNode variable in the statsDataTable, search the statsDataTable for siblings that are too close, and outputting a macro variable containing a list of the ParentNodes of the close siblings. Then:
PROC ARBOR INMODEL=savedTreeDataTable;
PRUNE &listOfNodes;
SAVE model=prunedTreeDataTable;
This is the simplist way I can think of doing it.
-Padraic
Don't miss out on SAS Innovate - Register now for the FREE Livestream!
Can't make it to Vegas? No problem! Watch our general sessions LIVE or on-demand starting April 17th. Hear from SAS execs, best-selling author Adam Grant, Hot Ones host Sean Evans, top tech journalist Kara Swisher, AI expert Cassie Kozyrkov, and the mind-blowing dance crew iLuminate! Plus, get access to over 20 breakout sessions.
Use this tutorial as a handy guide to weigh the pros and cons of these commonly used machine learning algorithms.
Find more tutorials on the SAS Users YouTube channel.