Decision Trees, HPSplit -- Pruning May Be Giving Away Valuable Info?

[NKormanik]

Wondering if pruning gives up too much valuable information.

Yes, the model becomes smaller, and supposedly more understandable.

But all those tips being whacked contain at least pairs of variables that ought to be acknowledged.

What are your thoughts on this conundrum?

Thanks!

Nicholas Kormanik

[PaigeMiller]

This is true in any modeling, the larger the model (in this case the more branches), the better it will fit (apparently).

But there is also a concept called overfitting, which would usually lead to smaller models, as overfitting is not good and essentially is fitting noise. Those extra branches that should get pruned may in fact be overfitting. How can you tell if something is overfit? Usually, by either crossvalidation, or by fitting the model to a training data set, and then evaluating its performance on validation (and test) data sets

[Reeza]

To be a contrarian, you can also be getting rid of the noise.