11-11-2016 01:44 PM
I'd like to use proc glmselect to compare ridge regresssion and LASSO on the same data. The documentation seems to say that selection=elasticnet with L1=0 is euivalent to ridge regression.
proc glmselect data=train plots=all; class private; model apps = private accept--grad_rate / selection=elasticnet(choose=cv l1=0 stop=cv); score data=test p out=ridge;
proc glmselect data=train plots=all; class private; model apps = private accept--grad_rate / selection=lasso(choose=cv stop=cv); score data=test p out=lasso;
The data is the College.csv file from http://www-bcf.usc.edu/~gareth/ISL/data.html, split into training and test sets.
The two models selected are exactly the same. (I also get the same model if I use elasticnet with L2=0.) Why is that? I expected ridge regression and LASSO to produce slightly different models since they use different constraints.
11-11-2016 03:02 PM
I think I found the answer. In the selection method options, L1=0 requires stop=L1. When I changed stop=cv to stop=l1, the ridge regression gave a different model than the LASSO.