topic Re: Interpreting Random Forest resulst in Statistical Procedures

Interpreting Random Forest resulst

jitb — Wed, 16 Jul 2025 16:10:10 GMT

Hi,

I am running a Proc HPForest model with several iterations based on different parameters. When I review the results, I find that the iteration (last row) that has the highest sensitivity, also has higher PredAll and PredOOB compared to other iterations. Any thoughts on this would be much appreciated. Thank you.

Obs	vars	leafsize	maxdepth	NTrees	NLeaves	PredAll	PredOob	MiscAll	MiscOob	LogLossAll	LogLossOob	Specificity	Sensitivity	Specificity_Test	Sensitivity_Test
1	4	10	20	100	13178	0.175	0.208	0.251	0.328	0.528	0.602	0.9315	0.56466	0.88214	0.660
2	3	10	20	100	11980	0.180	0.208	0.259	0.332	0.539	0.602	0.9370	0.59655	0.89163	0.692
3	4	5	20	100	25094	0.150	0.208	0.199	0.326	0.470	0.601	0.9500	0.45690	0.86802	0.620
4	3	5	20	100	22631	0.158	0.207	0.212	0.329	0.488	0.601	0.9495	0.48966	0.87518	0.636
5	3	10	20	150	18026	0.180	0.208	0.257	0.326	0.539	0.602	0.9385	0.59483	0.88952	0.684
6	2	10	15	75	7226	0.188	0.209	0.286	0.334	0.559	0.605	0.9455	0.68534	0.91609	0.730

Re: Interpreting Random Forest resulst

Ksharp — Thu, 17 Jul 2025 05:42:15 GMT

This NOTE would help you to understand concept Sensitivity and specificity.

https://support.sas.com/kb/24/170.html

Re: Interpreting Random Forest resulst

sbxkoenk — Thu, 17 Jul 2025 14:10:56 GMT

Hello ,

Measuring Prediction Error in PROC HPFOREST

The HPFOREST procedure computes the average square error measure of prediction error.
For a binary or nominal target, PROC HPFOREST also computes the misclassification rate and the log-loss.

See SAS Help Center: Measuring Prediction Error

Oob refers to Out-Of-Bag Estimates.
SAS Help Center: Example 7.1 Out-Of-Bag Estimate of Misclassification Rate

Ciao,

Koen

Re: Interpreting Random Forest resulst

jitb — Thu, 17 Jul 2025 15:35:29 GMT

Hi Koen,

My question was to explain a higher sensitivity with a corresponding higher pred error....see the last row of my table. I am trying to interpret this phenomenon and ultimately choose the best model in the table. Should I give more importance to sensitivity compared to the ASE? Thanks.

Re: Interpreting Random Forest resulst

jitb — Thu, 17 Jul 2025 17:06:42 GMT

My apologies...I made an error in calculating sensitivity. Higher sensitivity does correspond to lose ASE in my revised run. Sorry for the confusion 🙂