topic Brier score - Logistic Regression in Statistical Procedures https://communities.sas.com/t5/Statistical-Procedures/Brier-score-Logistic-Regression/m-p/207995#M11188 <HTML><HEAD></HEAD><BODY><P>Proc logistic returns brier score with fitstat option. My question - Can we set cutoff to check model calibration? What i understand it is sensitive to % of events. For example, for 50% event rate model, brier score less than 0.09 considers satisfactory. Does it exist for 10% event rate model? Does SAS generate scaled brier score?</P></BODY></HTML> Tue, 21 Jul 2015 21:48:58 GMT Ujjawal 2015-07-21T21:48:58Z Brier score - Logistic Regression https://communities.sas.com/t5/Statistical-Procedures/Brier-score-Logistic-Regression/m-p/207995#M11188 <HTML><HEAD></HEAD><BODY><P>Proc logistic returns brier score with fitstat option. My question - Can we set cutoff to check model calibration? What i understand it is sensitive to % of events. For example, for 50% event rate model, brier score less than 0.09 considers satisfactory. Does it exist for 10% event rate model? Does SAS generate scaled brier score?</P></BODY></HTML> Tue, 21 Jul 2015 21:48:58 GMT https://communities.sas.com/t5/Statistical-Procedures/Brier-score-Logistic-Regression/m-p/207995#M11188 Ujjawal 2015-07-21T21:48:58Z Re: Brier score - Logistic Regression https://communities.sas.com/t5/Statistical-Procedures/Brier-score-Logistic-Regression/m-p/207996#M11189 <HTML><HEAD></HEAD><BODY><P><SPAN style="color: #000000; font-family: arial, 'Arial Unicode MS', geneva, 'Lucida Grande', sans-serif; font-size: 13.4399995803833px; background-color: #ffffff;">In SAS website, it states that brier score </SPAN><SPAN style="color: #000000; font-family: arial, 'Arial Unicode MS', geneva, 'Lucida Grande', sans-serif; font-size: 13.4399995803833px; background-color: #ffffff;">is the weighted squared difference between the predicted probabilities and their observed response levels.</SPAN></P><P><SPAN style="color: #000000; font-family: arial, 'Arial Unicode MS', geneva, 'Lucida Grande', sans-serif; font-size: 13.4399995803833px; background-color: #ffffff;"><A class="active_link" href="http://support.sas.com/documentation/cdl/en/statug/63962/HTML/default/viewer.htm#statug_logistic_sect050.htm#statug.logistic.logdfitstat" title="http://support.sas.com/documentation/cdl/en/statug/63962/HTML/default/viewer.htm#statug_logistic_sect050.htm#statug.logistic.logdfitstat">SAS/STAT(R) 9.3 User's Guide</A></SPAN></P><P><SPAN style="color: #000000; font-family: arial, 'Arial Unicode MS', geneva, 'Lucida Grande', sans-serif; font-size: 13.4399995803833px; background-color: #ffffff;"><BR /></SPAN></P><P><SPAN style="color: #000000; font-family: arial, 'Arial Unicode MS', geneva, 'Lucida Grande', sans-serif; font-size: 13.4399995803833px; background-color: #ffffff;">Does it take into account the event rate (% of events)? Can i safely use cutoff .09? Brier score less than .09 is acceptable.<BR /></SPAN></P></BODY></HTML> Wed, 22 Jul 2015 11:25:08 GMT https://communities.sas.com/t5/Statistical-Procedures/Brier-score-Logistic-Regression/m-p/207996#M11189 Ujjawal 2015-07-22T11:25:08Z