About JTho

JTho · ‎10-16-2017

Colleagues: Running a decision tree in SAS-EM. I have a sample with 5.2 million records. The prevalence of the EVENT is 3% (156,000 events). It seems to me that a minimum terminal node sample size of 10, 50, 100, even 1,000 may be too small. Is anyone aware of a rule (or rule-of-thumb) for setting a reasonable min node for large datasets? As always, thank you for any suggestions/advice. -Josh

JTho · ‎09-26-2017

I am using SAS EM to grow a Decision Tree. I was asked by a reviewer if a "minimum difference" could be set at the terminal nodes. That is, could a threshold be set such that a parent node will not split unless the difference in the % event reaches a critical threshold value across nodes? For example: Prune back to the parent node, any terminal nodes where the % difference is less than 5%. The question was raised in the context of "clinically-meaningful" differences across terminal nodes vs. "statistically-meaningful" differences. Is this minimal difference something I could manually set in SAS-EM? Thank you! Josh

JTho · ‎06-05-2017

I am using HPSPLIT and working with very highly imbalanced database (3% had "event"). In this case, events are considered extremely costly so we are willing to trade off specificity (false positives) for sensitivity (false negatives). I have tried balancing the data (undersample non-events), but we are still missing too many events. Is there a more direct way of modifying the model to reflect the "high cost" of missing events? Priors, cost weights, etc? Thank you.

JTho · ‎05-17-2017

Thank you. Here it is. For Base SAS Software ... Custom version information: 9.4_M1 Image version information: 9.04.01M1P120413 For SAS/STAT ... Custom version information: 13.1 For SAS/GRAPH ... Custom version information: 9.4_M1 For SAS/ETS ... Custom version information: 13.1 For SAS/OR ... Custom version information: 13.1 For SAS/IML ... Custom version information: 13.1 For SAS/QC ... Custom version information: 13.1 For SAS/CONNECT ... Custom version information: 9.4_M1 For SAS Enterprise Miner ... Custom version information: 13.1 For SAS Integration Technologies ... Custom version information: 9.4_M1 For SAS/Secure 168-bit ... Custom version information: 9.4 Image version information: 9.04.01M0P061913 For High Performance Suite ... Custom version information: 2.1_M1 For SAS/ACCESS Interface to PC Files ... Custom version information: 9.4_M1 For SAS/ACCESS Interface to Microsoft SQL Server ... Custom version information: 9.4 Image version information: 9.04.01M0P061913 NOTE: PROCEDURE PRODUCT_STATUS used (Total process time): real time 0.25 seconds cpu time 0.03 seconds

JTho · ‎05-16-2017

I am running into the same issue...PROC HPSPLIT results are not displaying. I confirmed HPSPLIT is, in fact, building a classificaiton tree by outputting node statitistics (output nodestats=nstat). HPSPLIT is generating all sorts of output, but none appears in the Results window. I realize I'm a few weeks late to the discussion, but any new insights. Much appreciated.

Online Status	Offline
Date Last Visited	‎10-17-2017 09:33 AM

General CART advice: Rules for setting min terminal node sample sizes ...

Minimum difference across terminal nodes

HPSPLIT and rare events

Re: PROC HPSPLIT does not display expected report output in Enterprise...

Re: PROC HPSPLIT does not display expected report output in Enterprise...

Re: Minimum difference across terminal nodes

General CART advice: Rules for setting min terminal node sample sizes ...

Minimum difference across terminal nodes

HPSPLIT and rare events

Re: PROC HPSPLIT does not display expected report output in Enterprise...

Re: PROC HPSPLIT does not display expected report output in Enterprise...