Ridge in logistic regression?

[bollibompa]

Hi,

Has anyone performed a logistic ridge regression is SAS?

PROC REG procedure includes the RIDGE= statement but unfortunately not PROC LOGISTIC.

Thanks in advance!

Thomas

[SteveDenham]

I don't know what version of SAS/STAT you are using, but if it is 13.2 (the documentation I have open right now), it says that the default optimization technique with less than 40 parameters in PROC LOGISTIC is Newton-Raphson with ridging.  Can you supply some details on the model and proposed analysis to see if you need to invoke the TECHNIQUE= option in either the MODEL statement or the NLOPTIONS statement?

Steve Denham

[bollibompa]

Thanks for reply!

I use SAS/STAT 13.2.

What I want to perform is a penalized logistic regression in SAS (penaliz/shrink the size of the regression coefficients). There are different models for this (RIDGE, LASSO) and I have found this function in linear regression in SAS (PROC REG) but not in logistic regression (PROC LOGISTIC).

Very helpful if you can give input how this can be done with in SAS

Thanks!

Thomas

[lvm]

I am not aware of this functionality in a SAS procedure, certainly not in LOGISTIC or GENMOD. Maybe there are tricks to get this. Someone might have written a macro (certainly doable), but a quick search did not reveal anything.

[SteveDenham · Accepted solution]

Hi Thomas,

The next release of SAS/STAT (14.1) will provide LASSO techniques in PROC HPGENSELECT, which will be what you are looking for, I think.  It should be available later this year, if that helps at all.

Steve Denham

[bollibompa]

Thanks

Brilliant that it will be implemented in 14.1.

in the meantime, anyone seen a code/macro for  this problem?

/Thomas

[bollibompa]

I found a paper using shrinkage techniques in proc logistic "SUGI29-A new strategy of model buildning in proc logistic with automatic variable selection, validation, shrinkage and model averaging"

http://www2.sas.com/proceedings/sugi29/191-29.pdf

The paper says (on page 6):

"Shrunken regression coefficients are calculated by multiplication by the shrinkage factors (4) and (7). Note that 2logL(M) and 2logL(0) are the elements of the standard output of the SAS PROC LOGISTIC. So the heuristic linear shrinkage discussed above can be easily implemented in SAS PROC LOGISTIC by using ODS."

I am not so familiar with ODS and how this can be implemented in PROC LOGISTIC using ODS. Can you add your own formulas using ODS?

Unfortunately, the paper did not post any sas code examples.

Thanks for any help here!

/Thomas

[SteveDenham]

Add the following to your PROC LOGISTIC code:

ods output fitstatistics=fitstatistics:

This will create a dataset that gives -2 log-likelihood values for each step in the stepwise procedure.  -2log(M) is the log likelihood associated with the final model, -2log(0) is that associated with the intercept only model..  Once you have this dataset, you can readily calculate the lambda(AIC) shrinkage value given in the SUGI paper.

Steve Denham

[bollibompa]

Thank you for excellent help!

Using the fitstatistics I have now calculated the lambda(AIC) shrinkage value

May I ask a following question:

I use my logistic regression to calculate the propensity score for each observation in the dataset (PROC LOGISTIC, pred=p_score). Now when I have calulated shrinkage value a want to re-caluclate the propensity score based on the shrunken regression coefficients. Is that possible? Can all this be done in one step (adding formulas in the PROC LOGISTIC)?

Thanks again!

Thomas

[SteveDenham]

I don't see an obvious way, so it would be DATA step programming at this point (more reason to want SAS/STAT 14.1).

Steve Denham

[jwg]

Are you quite sure about this? I am in need of an implementation of this and have spent quite a bit of time chasing down this lead, but have found nothing.