BookmarkSubscribeRSS Feed
mcs
Obsidian | Level 7 mcs
Obsidian | Level 7

I'd like to use proc glmselect to compare ridge regresssion and LASSO on the same data.   The documentation seems to say that selection=elasticnet with L1=0 is euivalent to ridge regression.

 

proc glmselect data=train plots=all;
class private;
model apps = private accept--grad_rate / selection=elasticnet(choose=cv l1=0 stop=cv);
score data=test p out=ridge;
proc glmselect data=train plots=all;
class private;
model apps = private accept--grad_rate / selection=lasso(choose=cv stop=cv);
score data=test p out=lasso;

 

 

The data is the College.csv file from http://www-bcf.usc.edu/~gareth/ISL/data.html, split into training and test sets.

 

The two models selected are exactly the same.  (I also get the same model if I use elasticnet with L2=0.)  Why is that?  I expected ridge regression and LASSO to produce slightly different models since they use different constraints.

1 REPLY 1
mcs
Obsidian | Level 7 mcs
Obsidian | Level 7

I think I found the answer.  In the selection method options, L1=0 requires stop=L1.  When I changed stop=cv to stop=l1, the ridge regression gave a different model than the LASSO.

sas-innovate-2024.png

Don't miss out on SAS Innovate - Register now for the FREE Livestream!

Can't make it to Vegas? No problem! Watch our general sessions LIVE or on-demand starting April 17th. Hear from SAS execs, best-selling author Adam Grant, Hot Ones host Sean Evans, top tech journalist Kara Swisher, AI expert Cassie Kozyrkov, and the mind-blowing dance crew iLuminate! Plus, get access to over 20 breakout sessions.

 

Register now!

What is ANOVA?

ANOVA, or Analysis Of Variance, is used to compare the averages or means of two or more populations to better understand how they differ. Watch this tutorial for more.

Find more tutorials on the SAS Users YouTube channel.

Discussion stats
  • 1 reply
  • 2613 views
  • 0 likes
  • 1 in conversation