turn on suggestions

Auto-suggest helps you quickly narrow down your search results by suggesting possible matches as you type.

Showing results for

Find a Community

- Home
- /
- Analytics
- /
- Stat Procs
- /
- Ridge in logistic regression?

Topic Options

- Subscribe to RSS Feed
- Mark Topic as New
- Mark Topic as Read
- Float this Topic for Current User
- Bookmark
- Subscribe
- Printer Friendly Page

- Mark as New
- Bookmark
- Subscribe
- Subscribe to RSS Feed
- Permalink
- Email to a Friend
- Report Inappropriate Content

06-17-2015 05:03 AM

Hi,

Has anyone performed a logistic ridge regression is SAS?

PROC REG procedure includes the RIDGE= statement but unfortunately not PROC LOGISTIC.

Thanks in advance!

Thomas

Accepted Solutions

Solution

3 weeks ago

- Mark as New
- Bookmark
- Subscribe
- Subscribe to RSS Feed
- Permalink
- Email to a Friend
- Report Inappropriate Content

Posted in reply to bollibompa

06-22-2015 01:31 PM

Hi Thomas,

The next release of SAS/STAT (14.1) will provide LASSO techniques in PROC HPGENSELECT, which will be what you are looking for, I think. It should be available later this year, if that helps at all.

Steve Denham

All Replies

- Mark as New
- Bookmark
- Subscribe
- Subscribe to RSS Feed
- Permalink
- Email to a Friend
- Report Inappropriate Content

Posted in reply to bollibompa

06-17-2015 08:51 AM

I don't know what version of SAS/STAT you are using, but if it is 13.2 (the documentation I have open right now), it says that the default optimization technique with less than 40 parameters in PROC LOGISTIC is Newton-Raphson with ridging. Can you supply some details on the model and proposed analysis to see if you need to invoke the TECHNIQUE= option in either the MODEL statement or the NLOPTIONS statement?

Steve Denham

- Mark as New
- Bookmark
- Subscribe
- Subscribe to RSS Feed
- Permalink
- Email to a Friend
- Report Inappropriate Content

Posted in reply to SteveDenham

06-22-2015 04:09 AM

Thanks for reply!

I use SAS/STAT 13.2.

What I want to perform is a penalized logistic regression in SAS (penaliz/shrink the size of the regression coefficients). There are different models for this (RIDGE, LASSO) and I have found this function in linear regression in SAS (PROC REG) but not in logistic regression (PROC LOGISTIC).

Very helpful if you can give input how this can be done with in SAS

Thanks!

Thomas

- Mark as New
- Bookmark
- Subscribe
- Subscribe to RSS Feed
- Permalink
- Email to a Friend
- Report Inappropriate Content

Posted in reply to bollibompa

06-22-2015 11:47 AM

I am not aware of this functionality in a SAS procedure, certainly not in LOGISTIC or GENMOD. Maybe there are tricks to get this. Someone might have written a macro (certainly doable), but a quick search did not reveal anything.

Solution

3 weeks ago

- Mark as New
- Bookmark
- Subscribe
- Subscribe to RSS Feed
- Permalink
- Email to a Friend
- Report Inappropriate Content

Posted in reply to bollibompa

06-22-2015 01:31 PM

Hi Thomas,

The next release of SAS/STAT (14.1) will provide LASSO techniques in PROC HPGENSELECT, which will be what you are looking for, I think. It should be available later this year, if that helps at all.

Steve Denham

- Mark as New
- Bookmark
- Subscribe
- Subscribe to RSS Feed
- Permalink
- Email to a Friend
- Report Inappropriate Content

Posted in reply to SteveDenham

06-22-2015 03:01 PM

Thanks

Brilliant that it will be implemented in 14.1.

in the meantime, anyone seen a code/macro for this problem?

/Thomas

- Mark as New
- Bookmark
- Subscribe
- Subscribe to RSS Feed
- Permalink
- Email to a Friend
- Report Inappropriate Content

Posted in reply to bollibompa

06-23-2015 05:01 AM

I found a paper using shrinkage techniques in proc logistic "SUGI29-A new strategy of model buildning in proc logistic with automatic variable selection, validation, shrinkage and model averaging"

http://www2.sas.com/proceedings/sugi29/191-29.pdf

The paper says (on page 6):

*"Shrunken regression coefficients are calculated by multiplication by the shrinkage factors (4) and (7). Note that 2logL(M) and 2logL(0) are the elements of the standard output of the SAS PROC LOGISTIC. So the heuristic linear shrinkage discussed above can be easily implemented in SAS PROC LOGISTIC by using ODS."*

I am not so familiar with ODS and how this can be implemented in PROC LOGISTIC using ODS. Can you add your own formulas using ODS?

Unfortunately, the paper did not post any sas code examples.

Thanks for any help here!

/Thomas

- Mark as New
- Bookmark
- Subscribe
- Subscribe to RSS Feed
- Permalink
- Email to a Friend
- Report Inappropriate Content

Posted in reply to bollibompa

06-23-2015 11:21 AM

Add the following to your PROC LOGISTIC code:

ods output fitstatistics=fitstatistics:

This will create a dataset that gives -2 log-likelihood values for each step in the stepwise procedure. -2log(M) is the log likelihood associated with the final model, -2log(0) is that associated with the intercept only model.. Once you have this dataset, you can readily calculate the lambda(AIC) shrinkage value given in the SUGI paper.

Steve Denham

- Mark as New
- Bookmark
- Subscribe
- Subscribe to RSS Feed
- Permalink
- Email to a Friend
- Report Inappropriate Content

Posted in reply to SteveDenham

06-24-2015 05:40 AM

Thank you for excellent help!

Using the fitstatistics I have now calculated the lambda(AIC) shrinkage value

May I ask a following question:

I use my logistic regression to calculate the propensity score for each observation in the dataset (PROC LOGISTIC, pred=p_score). Now when I have calulated shrinkage value a want to re-caluclate the propensity score based on the shrunken regression coefficients. Is that possible? Can all this be done in one step (adding formulas in the PROC LOGISTIC)?

Thanks again!

Thomas

- Mark as New
- Bookmark
- Subscribe
- Subscribe to RSS Feed
- Permalink
- Email to a Friend
- Report Inappropriate Content

Posted in reply to bollibompa

06-24-2015 08:24 AM

I don't see an obvious way, so it would be DATA step programming at this point (more reason to want SAS/STAT 14.1).

Steve Denham

- Mark as New
- Bookmark
- Subscribe
- Subscribe to RSS Feed
- Permalink
- Email to a Friend
- Report Inappropriate Content

Posted in reply to SteveDenham

03-22-2017 09:06 AM

Are you quite sure about this? I am in need of an implementation of this and have spent quite a bit of time chasing down this lead, but have found nothing.