07-21-2015 05:48 PM
Proc logistic returns brier score with fitstat option. My question - Can we set cutoff to check model calibration? What i understand it is sensitive to % of events. For example, for 50% event rate model, brier score less than 0.09 considers satisfactory. Does it exist for 10% event rate model? Does SAS generate scaled brier score?
07-22-2015 07:25 AM
In SAS website, it states that brier score is the weighted squared difference between the predicted probabilities and their observed response levels.
Does it take into account the event rate (% of events)? Can i safely use cutoff .09? Brier score less than .09 is acceptable.