Using %nlmeans for Firth or exact logistic regression

[tatami]

Due to sparse data and separation issue, I am running Firth or exact logistic regression instead of regular logistic regression. The statistics of interest is risk difference instead of odds ratio. I noticed that I can run "%nlmeans" with output from proc logistic firth but there is warning message "WARNING: The final Hessian matrix is not positive definite, and therefore the estimated covariance matrix is not full rank and may be

unreliable. The variance of some parameter estimates is zero or some parameters are linearly related to other parameters."  Are the output still valid? If not, is there any alternative? How about obtaining risk difference from exact logistic regression?

[StatDave]

As mentioned in the Warning, the results may not be reliable. You should try to modify your model in some way so that the Warning does not occur, or possibly try an alternative analysis method. For example, if all of your predictors are categorical, you might be able to use a stratified approach in PROC FREQ or PROC MULTTEST as shown in this note. With the nonmodel-based stratified analysis in PROC FREQ, you could use the COMMONRISKDIFF option to get an estimate of the risk difference. Exact analysis is another option if the data set is small. However, since the LSMEANS and ESTIMATE statements cannot be used with exact analysis, the NLMeans macro, which uses results from one of these statements, cannot be used with the exact analysis results.

[tatami]

Thanks for the reply. Soon after I posted this question, I saw the documentation here that states that the warning sign can be ignored. Running the Firth logistic regression itself does not produce any warning or error messages. The model converges. Yes, all of the covariates are categorical and has sufficient counts. The number of events are generally low hence the use of Firth. So my question on the validity of NLMeans output for Firth logistic regression still stands.

[StatDave]

Ok, so if that message is coming from NLMeans, not PROC LOGISTIC, then yes, it is ignorable and just occurs because of the use of GLM parameterization. In that case, the results from LSMEANS or ESTIMATE statement(s) can be used with NLMeans and the results should correctly be based on the Firth-based results.

[StatDave]

Also, while these modeling approaches might run into similar estimation problems, you could try the log-linked binomial model or the modified Poisson model as described in this note on estimating the relative risk.