topic Decision Trees, HPSplit -- Pruning May Be Giving Away Valuable Info? in SAS Procedures

Decision Trees, HPSplit -- Pruning May Be Giving Away Valuable Info?

NKormanik — Sun, 01 Mar 2020 10:01:55 GMT

Wondering if pruning gives up too much valuable information.

Yes, the model becomes smaller, and supposedly more understandable.

But all those tips being whacked contain at least pairs of variables that ought to be acknowledged.

What are your thoughts on this conundrum?

Thanks!

Nicholas Kormanik

Re: Decision Trees, HPSplit -- Pruning May Be Giving Away Valuable Info?

PaigeMiller — Sun, 01 Mar 2020 11:48:02 GMT

This is true in any modeling, the larger the model (in this case the more branches), the better it will fit (apparently).

But there is also a concept called overfitting, which would usually lead to smaller models, as overfitting is not good and essentially is fitting noise. Those extra branches that should get pruned may in fact be overfitting. How can you tell if something is overfit? Usually, by either crossvalidation, or by fitting the model to a training data set, and then evaluating its performance on validation (and test) data sets

Re: Decision Trees, HPSplit -- Pruning May Be Giving Away Valuable Info?

Reeza — Sun, 01 Mar 2020 22:23:40 GMT

To be a contrarian, you can also be getting rid of the noise.