BookmarkSubscribeRSS Feed
🔒 This topic is solved and locked. Need further help from the community? Please sign in and ask a new question.
bollibompa
Quartz | Level 8

Hi,

Has anyone performed a logistic ridge regression is SAS?

PROC REG procedure includes the RIDGE= statement but unfortunately not PROC LOGISTIC.

Thanks in advance!

Thomas

1 ACCEPTED SOLUTION

Accepted Solutions
SteveDenham
Jade | Level 19

Hi Thomas,

The next release of SAS/STAT (14.1) will provide LASSO techniques in PROC HPGENSELECT, which will be what you are looking for, I think.  It should be available later this year, if that helps at all.

Steve Denham

View solution in original post

10 REPLIES 10
SteveDenham
Jade | Level 19

I don't know what version of SAS/STAT you are using, but if it is 13.2 (the documentation I have open right now), it says that the default optimization technique with less than 40 parameters in PROC LOGISTIC is Newton-Raphson with ridging.  Can you supply some details on the model and proposed analysis to see if you need to invoke the TECHNIQUE= option in either the MODEL statement or the NLOPTIONS statement?

Steve Denham

bollibompa
Quartz | Level 8

Thanks for reply!

I use SAS/STAT 13.2.

What I want to perform is a penalized logistic regression in SAS (penaliz/shrink the size of the regression coefficients). There are different models for this (RIDGE, LASSO) and I have found this function in linear regression in SAS (PROC REG) but not in logistic regression (PROC LOGISTIC).

Very helpful if you can give input how this can be done with in SAS

Thanks!

Thomas

lvm
Rhodochrosite | Level 12 lvm
Rhodochrosite | Level 12

I am not aware of this functionality in a SAS procedure, certainly not in LOGISTIC or GENMOD. Maybe there are tricks to get this. Someone might have written a macro (certainly doable), but a quick search did not reveal anything.

SteveDenham
Jade | Level 19

Hi Thomas,

The next release of SAS/STAT (14.1) will provide LASSO techniques in PROC HPGENSELECT, which will be what you are looking for, I think.  It should be available later this year, if that helps at all.

Steve Denham

bollibompa
Quartz | Level 8

Thanks

Brilliant that it will be implemented in 14.1.

in the meantime, anyone seen a code/macro for  this problem?

/Thomas

bollibompa
Quartz | Level 8

I found a paper using shrinkage techniques in proc logistic "SUGI29-A new strategy of model buildning in proc logistic with automatic variable selection, validation, shrinkage and model averaging"

http://www2.sas.com/proceedings/sugi29/191-29.pdf

The paper says (on page 6):

"Shrunken regression coefficients are calculated by multiplication by the shrinkage factors (4) and (7). Note that 2logL(M) and 2logL(0) are the elements of the standard output of the SAS PROC LOGISTIC. So the heuristic linear shrinkage discussed above can be easily implemented in SAS PROC LOGISTIC by using ODS."


I am not so familiar with ODS and how this can be implemented in PROC LOGISTIC using ODS. Can you add your own formulas using ODS?

Unfortunately, the paper did not post any sas code examples.

Thanks for any help here!

/Thomas

SteveDenham
Jade | Level 19

Add the following to your PROC LOGISTIC code:

ods output fitstatistics=fitstatistics:

This will create a dataset that gives -2 log-likelihood values for each step in the stepwise procedure.  -2log(M) is the log likelihood associated with the final model, -2log(0) is that associated with the intercept only model..  Once you have this dataset, you can readily calculate the lambda(AIC) shrinkage value given in the SUGI paper.

Steve Denham

bollibompa
Quartz | Level 8

Thank you for excellent help!

Using the fitstatistics I have now calculated the lambda(AIC) shrinkage value

May I ask a following question:

I use my logistic regression to calculate the propensity score for each observation in the dataset (PROC LOGISTIC, pred=p_score). Now when I have calulated shrinkage value a want to re-caluclate the propensity score based on the shrunken regression coefficients. Is that possible? Can all this be done in one step (adding formulas in the PROC LOGISTIC)?

Thanks again!

Thomas

SteveDenham
Jade | Level 19

I don't see an obvious way, so it would be DATA step programming at this point (more reason to want SAS/STAT 14.1).

Steve Denham

jwg
Calcite | Level 5 jwg
Calcite | Level 5

Are you quite sure about this? I am in need of an implementation of this and have spent quite a bit of time chasing down this lead, but have found nothing.

sas-innovate-2024.png

Join us for SAS Innovate April 16-19 at the Aria in Las Vegas. Bring the team and save big with our group pricing for a limited time only.

Pre-conference courses and tutorials are filling up fast and are always a sellout. Register today to reserve your seat.

 

Register now!

What is ANOVA?

ANOVA, or Analysis Of Variance, is used to compare the averages or means of two or more populations to better understand how they differ. Watch this tutorial for more.

Find more tutorials on the SAS Users YouTube channel.

Discussion stats
  • 10 replies
  • 7203 views
  • 1 like
  • 4 in conversation