Minimum difference across terminal nodes

JTho — Tue, 26 Sep 2017 15:04:35 GMT

I am using SAS EM to grow a Decision Tree. I was asked by a reviewer if a "minimum difference" could be set at the terminal nodes. That is, could a threshold be set such that a parent node will not split unless the difference in the % event reaches a critical threshold value across nodes?

For example: Prune back to the parent node, any terminal nodes where the % difference is less than 5%.

The question was raised in the context of "clinically-meaningful" differences across terminal nodes vs. "statistically-meaningful" differences.

Is this minimal difference something I could manually set in SAS-EM?

Thank you!

Josh

Re: Minimum difference across terminal nodes

PadraicGNeville — Thu, 28 Sep 2017 15:19:47 GMT

No: The procedure that underlies the EM decision tree node does not have an option to prune close siblings. One would have to resort to advanced SAS data step programming, outputing the nodes statistics, searching for leaves that are too close, and pruning their parents. Some key elements might be:

PROC ARBOR INMODEL = savedTreeDataTable;

SAVE STATSBYNODE= statsDataTable TOPOLOGY=topologyDataTable.

Write data step code to use the topologyDataTable to add a ParentNode variable in the statsDataTable, search the statsDataTable for siblings that are too close, and outputting a macro variable containing a list of the ParentNodes of the close siblings. Then:

PROC ARBOR INMODEL=savedTreeDataTable;

PRUNE &listOfNodes;

SAVE model=prunedTreeDataTable;

This is the simplist way I can think of doing it.

-Padraic

topic Re: Minimum difference across terminal nodes in SAS Data Science

Minimum difference across terminal nodes

Re: Minimum difference across terminal nodes