Programming the statistical procedures from SAS

Ridge Regression and LASSO

Reply
Contributor mcs
Contributor
Posts: 21

Ridge Regression and LASSO

I'd like to use proc glmselect to compare ridge regresssion and LASSO on the same data.   The documentation seems to say that selection=elasticnet with L1=0 is euivalent to ridge regression.

 

proc glmselect data=train plots=all;
class private;
model apps = private accept--grad_rate / selection=elasticnet(choose=cv l1=0 stop=cv);
score data=test p out=ridge;
proc glmselect data=train plots=all;
class private;
model apps = private accept--grad_rate / selection=lasso(choose=cv stop=cv);
score data=test p out=lasso;

 

 

The data is the College.csv file from http://www-bcf.usc.edu/~gareth/ISL/data.html, split into training and test sets.

 

The two models selected are exactly the same.  (I also get the same model if I use elasticnet with L2=0.)  Why is that?  I expected ridge regression and LASSO to produce slightly different models since they use different constraints.

Contributor mcs
Contributor
Posts: 21

Re: Ridge Regression and LASSO

I think I found the answer.  In the selection method options, L1=0 requires stop=L1.  When I changed stop=cv to stop=l1, the ridge regression gave a different model than the LASSO.

Ask a Question
Discussion stats
  • 1 reply
  • 741 views
  • 0 likes
  • 1 in conversation