topic Re: Undersampling in PROC PROC HPSPLIT / Adjusting for prior probablities in Statistical Procedures

Undersampling in PROC PROC HPSPLIT / Adjusting for prior probablities

lukholoman — Mon, 10 Nov 2025 15:14:00 GMT

Greetings, I am working with imbalanced data where my target is 1,465 "Fail" (event) and the non-event is 58,744. I have decided to undersample the data. I wanted to then ask, how do I adjust for prior probabilities in the PROC HPSPLIT? Is it supported or not?

Re: Undersampling in PROC PROC HPSPLIT / Adjusting for prior probablities

StatDave — Mon, 10 Nov 2025 16:28:45 GMT

I do not know that the tree-based method implemented in PROC HPSPLIT has any provision to specifically handle under- or oversampling of the response. However, this issue often comes up when using logistic modeling of a binary response and there are methods discussed in this note for dealing with that. While not designed for the tree-based method, you could consider using the weighting method presented in the note since HPSPLIT does support a WEIGHT statement.

Re: Undersampling in PROC PROC HPSPLIT / Adjusting for prior probablities

sbxkoenk — Mon, 10 Nov 2025 16:55:18 GMT

@StatDave wrote:

... and there are methods discussed in this note for dealing with that.

Topic = adjusting the posterior probabilities for the real priors after under-sampling the majority class in binary classification.

If you want some additional prose on that topic and on that note (mentioned by @StatDave) , see here :

Ciao,

Koen

Re: Undersampling in PROC PROC HPSPLIT / Adjusting for prior probablities

Ksharp — Tue, 11 Nov 2025 07:07:09 GMT

As @StatDave showed you , adjust prior probability, check this option:

   proc logistic data=out;
        model y(event="1")=x;
        score data=sub prior=priors out=out2;