Hello,
I am looking at the Output in Results after running a Model Comaprison node for four models. I understand how lift/gain/%response is calculated across deciles but was wondering how these values are calculated for the 'whole' model (see below - apologies about formatting).
Thanks,
Martin
Data Role=Valid
Statistics Reg Tree Neural Tree2
Valid: Kolmogorov-Smirnov Statistic 0.41 0.37 0.42 0.37
Valid: Average Squared Error 0.12 0.12 0.13 0.13
Valid: Roc Index 0.76 0.72 0.77 0.72
Valid: Average Error Function 0.39 . 0.40 .
Valid: Bin-Based Two-Way Kolmogorov-Smirnov Probability Cutoff 0.13 0.10 0.17 0.22
Valid: Cumulative Percent Captured Response 27.43 27.54 21.71 21.26
Valid: Percent Captured Response 13.14 10.91 8.00 10.53
Valid: Divisor for VASE 2100.00 2100.00 2100.00 2100.00
Valid: Error Function 820.57 . 833.16 .
Valid: Gain 174.29 175.41 117.14 112.60
Valid: Gini Coefficient 0.52 0.43 0.53 0.43
Valid: Bin-Based Two-Way Kolmogorov-Smirnov Statistic 0.39 0.36 0.41 0.36
Valid: Kolmogorov-Smirnov Probability Cutoff 0.12 0.08 0.10 0.08
Valid: Cumulative Lift 2.74 2.75 2.17 2.13
Valid: Lift 2.65 2.20 1.62 2.13
Valid: Maximum Absolute Error 0.97 0.93 0.99 0.93
Valid: Misclassification Rate 0.17 0.16 0.17 0.17
Valid: Mean Squared Error 0.12 . 0.13 .
Valid: Sum of Frequencies 1050.00 1050.00 1050.00 1050.00
Valid: Root Average Squared Error 0.35 0.35 0.35 0.35
Valid: Cumulative Percent Response 45.71 45.90 36.19 35.43
Valid: Percent Response 44.23 36.70 26.92 35.43
Valid: Root Mean Squared Error 0.35 . 0.35 .
Valid: Sum of Squared Errors 256.65 261.89 263.95 264.27
Valid: Sum of Case Weights Times Freq 2100.00 . 2100.00 .
Valid: Number of Wrong Classifications . . 181.00 .
Good question - it's not very clear in the output. It's actually not for the whole model, but reporting the lift/gain-like statistics at depth 10 in the Statistics Comparison table.
Good question - it's not very clear in the output. It's actually not for the whole model, but reporting the lift/gain-like statistics at depth 10 in the Statistics Comparison table.
It's finally time to hack! Remember to visit the SAS Hacker's Hub regularly for news and updates.
Use this tutorial as a handy guide to weigh the pros and cons of these commonly used machine learning algorithms.
Find more tutorials on the SAS Users YouTube channel.