BookmarkSubscribeRSS Feed
JerryLee
Calcite | Level 5

I am trying to optimize a L1-penalized log-likelihood function (a.k.a. LASSO) where a L1-penalized term was added to the negative loglikelihood function. I tried to use call NLPNRA and call NLPNMS. But they did not work. I am not sure whether it is because the penalized term involves terms of absolute values.

I know that there have been a great deal of theoretical developments to increase the efficiency of optimizing L1-penalized log-likelihood function, e.g., Efron el al. (2004) LEAST ANGLE REGRESSION. The Annals of Statistics; and Li and Osher (2010) Coordinate Descent Optimization for ℓ 1 Minimization with Application to Compressed Sensing; a Greedy Algorithm. Do you guys know whether IML has incorporated any of the new developments for optimizing a L1-penalized loglikelihood function? Thanks a lot!

3 REPLIES 3
1zmm
Quartz | Level 8

I don't know about PROC IML, but PROC GLMSELECT can use either the LASSO or the Least Angle Regression method to select variables.

Rick_SAS
SAS Super FREQ

As 1zmm says, if this is truly the lasso objective function, use PROC GLMSELECT.


To answer your question, no, SAS/IML doesn't have any special-purpose optimization routines for L1 objective functions. In practice, people often formulate the problem in order to avoid the (slow) optimization. Instead, they develop techniques that enable them to solve the problem directly. For example, a popular technique for dealing with L1 optimization is called "coordinate descent." A reference is

http://www.ncbi.nlm.nih.gov/pmc/articles/PMC2929880/

In the article, the authors want to optimize Eqn 1, which has an L1 penalty term.  They show that you can solve a series of one-dimensional problems whose solution is given by Eqn 5.  (Details in http://www.jstor.org/stable/10.2307/4537438)

JerryLee
Calcite | Level 5

Thanks a lot for providing the reference papers.

sas-innovate-2024.png

📢

ANNOUNCEMENT

The early bird rate has been extended! Register by March 18 for just $695 - $100 off the standard rate.

 

Check out the agenda and get ready for a jam-packed event featuring workshops, super demos, breakout sessions, roundtables, inspiring keynotes and incredible networking events. 

 

Register now!

Multiple Linear Regression in SAS

Learn how to run multiple linear regression models with and without interactions, presented by SAS user Alex Chaplin.

Find more tutorials on the SAS Users YouTube channel.

From The DO Loop
Want more? Visit our blog for more articles like these.
Discussion stats
  • 3 replies
  • 1089 views
  • 6 likes
  • 3 in conversation