I am trying to optimize a L1-penalized log-likelihood function (a.k.a. LASSO) where a L1-penalized term was added to the negative loglikelihood function. I tried to use call NLPNRA and call NLPNMS. But they did not work. I am not sure whether it is because the penalized term involves terms of absolute values.
I know that there have been a great deal of theoretical developments to increase the efficiency of optimizing L1-penalized log-likelihood function, e.g., Efron el al. (2004) LEAST ANGLE REGRESSION. The Annals of Statistics; and Li and Osher (2010) Coordinate Descent Optimization for ℓ 1 Minimization with Application to Compressed Sensing; a Greedy Algorithm. Do you guys know whether IML has incorporated any of the new developments for optimizing a L1-penalized loglikelihood function? Thanks a lot!
As 1zmm says, if this is truly the lasso objective function, use PROC GLMSELECT.
To answer your question, no, SAS/IML doesn't have any special-purpose optimization routines for L1 objective functions. In practice, people often formulate the problem in order to avoid the (slow) optimization. Instead, they develop techniques that enable them to solve the problem directly. For example, a popular technique for dealing with L1 optimization is called "coordinate descent." A reference is
In the article, the authors want to optimize Eqn 1, which has an L1 penalty term. They show that you can solve a series of one-dimensional problems whose solution is given by Eqn 5. (Details in http://www.jstor.org/stable/10.2307/4537438)
Registration is open! SAS is returning to Vegas for an AI and analytics experience like no other! Whether you're an executive, manager, end user or SAS partner, SAS Innovate is designed for everyone on your team. Register for just $495 by 12/31/2023.
If you are interested in speaking, there is still time to submit a session idea. More details are posted on the website.