Hi, I'm using the model comparison in SAS/VS. It does some strange things. I have an RF model with reported Validation ASE of 197,611. I have a Gradient Boosting (DSG) model with a reported Validation ASE of 199,686. After running this through model comparison I get a validation ASE of 196,927 for the RF model and 202,275 for the DSG model. These numbers vary with the number of models being in the Model Comparison.
Of a larger concern is that the Variable importance is completely changed. The variable that in the DSG model is the second most important has suddenly become the 7 most important after Model comparison. Also the variable importance HeatMap is inconsistent with the variable importance table.
Use this tutorial as a handy guide to weigh the pros and cons of these commonly used machine learning algorithms.
Find more tutorials on the SAS Users YouTube channel.
-
392 views
-
0 likes
-
-
