optimization of L1-penalized log-likelihood function

[JerryLee]

I am trying to optimize a L1-penalized log-likelihood function (a.k.a. LASSO) where a L1-penalized term was added to the negative loglikelihood function. I tried to use call NLPNRA and call NLPNMS. But they did not work. I am not sure whether it is because the penalized term involves terms of absolute values.

I know that there have been a great deal of theoretical developments to increase the efficiency of optimizing L1-penalized log-likelihood function, e.g., Efron el al. (2004) LEAST ANGLE REGRESSION. The Annals of Statistics; and Li and Osher (2010) Coordinate Descent Optimization for ℓ 1 Minimization with Application to Compressed Sensing; a Greedy Algorithm. Do you guys know whether IML has incorporated any of the new developments for optimizing a L1-penalized loglikelihood function? Thanks a lot!

[1zmm]

I don't know about PROC IML, but PROC GLMSELECT can use either the LASSO or the Least Angle Regression method to select variables.

[Rick_SAS]

As 1zmm says, if this is truly the lasso objective function, use PROC GLMSELECT.

To answer your question, no, SAS/IML doesn't have any special-purpose optimization routines for L1 objective functions. In practice, people often formulate the problem in order to avoid the (slow) optimization. Instead, they develop techniques that enable them to solve the problem directly. For example, a popular technique for dealing with L1 optimization is called "coordinate descent." A reference is

http://www.ncbi.nlm.nih.gov/pmc/articles/PMC2929880/

In the article, the authors want to optimize Eqn 1, which has an L1 penalty term.  They show that you can solve a series of one-dimensional problems whose solution is given by Eqn 5.  (Details in http://www.jstor.org/stable/10.2307/4537438)

[JerryLee]

Thanks a lot for providing the reference papers.