Interpreting Random Forest resulst

[jitb]

Hi,

I am running a Proc HPForest model with several iterations based on different parameters. When I review the results, I find that the iteration (last row) that has the highest sensitivity, also has higher PredAll and PredOOB compared to other iterations. Any thoughts on this would be much appreciated. Thank you.

Obs	vars	leafsize	maxdepth	NTrees	NLeaves	PredAll	PredOob	MiscAll	MiscOob	LogLossAll	LogLossOob	Specificity	Sensitivity	Specificity_Test	Sensitivity_Test
1	4	10	20	100	13178	0.175	0.208	0.251	0.328	0.528	0.602	0.9315	0.56466	0.88214	0.660
2	3	10	20	100	11980	0.180	0.208	0.259	0.332	0.539	0.602	0.9370	0.59655	0.89163	0.692
3	4	5	20	100	25094	0.150	0.208	0.199	0.326	0.470	0.601	0.9500	0.45690	0.86802	0.620
4	3	5	20	100	22631	0.158	0.207	0.212	0.329	0.488	0.601	0.9495	0.48966	0.87518	0.636
5	3	10	20	150	18026	0.180	0.208	0.257	0.326	0.539	0.602	0.9385	0.59483	0.88952	0.684
6	2	10	15	75	7226	0.188	0.209	0.286	0.334	0.559	0.605	0.9455	0.68534	0.91609	0.730

[Ksharp]

This NOTE would help you to understand concept Sensitivity and specificity.

https://support.sas.com/kb/24/170.html

[sbxkoenk]

Hello ,

Measuring Prediction Error in PROC HPFOREST

• The HPFOREST procedure computes the average square error measure of prediction error.
• For a binary or nominal target, PROC HPFOREST also computes the misclassification rate and the log-loss.

See SAS Help Center: Measuring Prediction Error

Oob refers to  Out-Of-Bag Estimates.
SAS Help Center: Example 7.1 Out-Of-Bag Estimate of Misclassification Rate

Ciao, 

Koen

[jitb]

Hi Koen,

My question was to explain a higher sensitivity with a corresponding higher pred error....see the last row of my table. I am trying to interpret this phenomenon and ultimately choose the best model in the table. Should I give more importance to sensitivity compared to the ASE? Thanks.

[jitb · Accepted solution]

My apologies...I made an error in calculating sensitivity. Higher sensitivity does correspond to lose ASE in my revised run. Sorry for the confusion 🙂