turn on suggestions

Auto-suggest helps you quickly narrow down your search results by suggesting possible matches as you type.

Showing results for

Find a Community

Topic Options

- Subscribe to RSS Feed
- Mark Topic as New
- Mark Topic as Read
- Float this Topic for Current User
- Bookmark
- Subscribe
- Printer Friendly Page

- Mark as New
- Bookmark
- Subscribe
- Subscribe to RSS Feed
- Permalink
- Email to a Friend
- Report Inappropriate Content

04-04-2013 04:03 PM

I am trying to optimize a L1-penalized log-likelihood function (a.k.a. LASSO) where a L1-penalized term was added to the negative loglikelihood function. I tried to use call NLPNRA and call NLPNMS. But they did not work. I am not sure whether it is because the penalized term involves terms of absolute values.

I know that there have been a great deal of theoretical developments to increase the efficiency of optimizing L1-penalized log-likelihood function, e.g., Efron el al. (2004) LEAST ANGLE REGRESSION. The Annals of Statistics; and Li and Osher (2010) Coordinate Descent Optimization for ℓ 1 Minimization with Application to Compressed Sensing; a Greedy Algorithm. Do you guys know whether IML has incorporated any of the new developments for optimizing a L1-penalized loglikelihood function? Thanks a lot!

- Mark as New
- Bookmark
- Subscribe
- Subscribe to RSS Feed
- Permalink
- Email to a Friend
- Report Inappropriate Content

Posted in reply to JerryLee

04-07-2013 09:52 AM

I don't know about PROC IML, but PROC GLMSELECT can use either the LASSO or the Least Angle Regression method to select variables.

- Mark as New
- Bookmark
- Subscribe
- Subscribe to RSS Feed
- Permalink
- Email to a Friend
- Report Inappropriate Content

Posted in reply to JerryLee

04-08-2013 10:45 AM

As 1zmm says, if this is truly the lasso objective function, use PROC GLMSELECT.

To answer your question, no, SAS/IML doesn't have any special-purpose optimization routines for L1 objective functions. In practice, people often formulate the problem in order to avoid the (slow) optimization. Instead, they develop techniques that enable them to solve the problem directly. For example, a popular technique for dealing with L1 optimization is called "coordinate descent." A reference is

http://www.ncbi.nlm.nih.gov/pmc/articles/PMC2929880/

In the article, the authors want to optimize Eqn 1, which has an L1 penalty term. They show that you can solve a series of one-dimensional problems whose solution is given by Eqn 5. (Details in http://www.jstor.org/stable/10.2307/4537438)

- Mark as New
- Bookmark
- Subscribe
- Subscribe to RSS Feed
- Permalink
- Email to a Friend
- Report Inappropriate Content

Posted in reply to Rick_SAS

04-10-2013 04:05 PM

Thanks a lot for providing the reference papers.