Ridge Regression and LASSO

mcs — Fri, 11 Nov 2016 18:44:15 GMT

I'd like to use proc glmselect to compare ridge regresssion and LASSO on the same data. The documentation seems to say that selection=elasticnet with L1=0 is euivalent to ridge regression.

proc glmselect data=train plots=all;
class private;
model apps = private accept--grad_rate / selection=elasticnet(choose=cv l1=0 stop=cv);
score data=test p out=ridge;

proc glmselect data=train plots=all;
class private;
model apps = private accept--grad_rate / selection=lasso(choose=cv stop=cv);
score data=test p out=lasso;

The data is the College.csv file from http://www-bcf.usc.edu/~gareth/ISL/data.html, split into training and test sets.

The two models selected are exactly the same. (I also get the same model if I use elasticnet with L2=0.) Why is that? I expected ridge regression and LASSO to produce slightly different models since they use different constraints.

Re: Ridge Regression and LASSO

mcs — Fri, 11 Nov 2016 20:02:46 GMT

I think I found the answer. In the selection method options, L1=0 requires stop=L1. When I changed stop=cv to stop=l1, the ridge regression gave a different model than the LASSO.

topic Re: Ridge Regression and LASSO in Statistical Procedures

Ridge Regression and LASSO

Re: Ridge Regression and LASSO