Firth bias correction

[Minaadolf]

can any one explain firth bias correction for model separation in logistic regression

[StatDave]

It simply adds a quantity (based on the information matrix) to the likelihood function and the gradients. The modified likelihood is then maximized in the usual way. See "Firth's Bias-Reduced Penalized Likelihood" in the Details section of the PROC LOGISTIC documentation. As stated in the description of the FIRTH option: Firth’s penalized maximum likelihood estimation is used to reduce bias in the parameter estimates (Heinze and Schemper 2002; Firth 1993). This method is useful in cases of separability, as often occurs when the event is rare, and is an alternative to performing an exact logistic regression. It often is a first alternative when there are problems with getting the model converge because of sparseness in the data and is far more feasible than the exact estimation method available via the EXACT statement but which is extremely resource (time, memory) intensive. 

[sbxkoenk]

FIRTH option in MODEL statement

FIRTH  = Firth’s bias-reducing penalized likelihood method ... available for binary response models only !

FIRTH option in MODEL statement performs Firth’s penalized maximum likelihood estimation to reduce bias in the parameter estimates (Heinze and Schemper 2002; Firth 1993). This method is useful in cases of separability, as often occurs when the event is rare, and is an alternative to performing an exact logistic regression. See the section Firth’s Bias-Reducing Penalized Likelihood for more information.

( Note: The intercept-only log likelihood is modified by using the full-model Hessian, computed with the slope parameters equal to zero. When fitting a model and scoring a data set in the same PROC LOGISTIC step, the model is fit using Firth’s penalty for parameter estimation purposes, but the penalty is not applied to the scored log likelihood. )

• Heinze, G., and Schemper, M. (2002). “A Solution to the Problem of Separation in Logistic Regression.” Statistics in Medicine 21:2409–2419.

• Firth, D. (1993). “Bias Reduction of Maximum Likelihood Estimates.” Biometrika 80:27–38.

SAS Help Center: Firth’s Penalized Likelihood Compared with Other Approaches

This FIRTH method has the advantage of being much faster and less memory intensive than exact (Exact Conditional Logistic Regression) algorithms, but it might not converge to a solution.

Ciao, Koen