I have a simple gradient boosting model (maximum branch = 2 , maximum depth = 1 {Adaboost} ) in e-miner(v 14.1) with binary target and mostly interval inputs(~500 variables). I will be choosing variables if the variable importance > 0.05 for both training and validation datasets. However, I am trying to understand the mathematics behind how the "variable importance" is calculated. I read the documentation (decision tree variable importance ) but its very vague. I was wondering if anyone could shed light on how it is calculated with a simple example? It will be very helpful.
feature importance for a single decision tree - the amount that each attribute split point improves the performance measure, weighted by the number of observations the node is responsible for. The performance measure may be the purity (Gini index) used to select the split points, or another more specific error function.
overall feature importance - feature importances averaged across all of the the decision trees within the model.
I am attaching the screenshot from SAS Enterprise miner Reference documentation 14.3 where you can find the official computation description.
April 27 – 30 | Gaylord Texan | Grapevine, Texas
Walk in ready to learn. Walk out ready to deliver. This is the data and AI conference you can't afford to miss.
Register now and lock in 2025 pricing—just $495!
Still thinking about your presentation idea? The submission deadline has been extended to Friday, Nov. 14, at 11:59 p.m. ET.
Use this tutorial as a handy guide to weigh the pros and cons of these commonly used machine learning algorithms.
Find more tutorials on the SAS Users YouTube channel.