topic Re: proc gmlselect + elastic net without feature selection in Statistical Procedures

proc gmlselect + elastic net without feature selection

csetzkorn — Mon, 23 Apr 2018 09:01:03 GMT

is it possible to use:

selection=ELASTICNET ...

in PROC GLMSELECT so that no feature selection is performed (i.e. all featured are 'forced' into model)?

Re: proc gmlselect + elastic net without feature selection

Rick_SAS — Mon, 23 Apr 2018 09:45:38 GMT

I don't think so. If you want all variables in the model, use SELECTION=NONE to get the OLS estimates. But I don't think you can get the elastic net estimates for the full model.

Re: proc gmlselect + elastic net without feature selection

csetzkorn — Mon, 23 Apr 2018 10:06:57 GMT

OK thanks. Wanted to use elastic net as there are correlations between at least 2 variables ... Do you know if, in this case, OLS estimates could still be used purely for prediction rather than interpretation? Thanks.

Re: proc gmlselect + elastic net without feature selection

Rick_SAS — Mon, 23 Apr 2018 10:20:52 GMT

It depends on how correlated the variables are. Strong correlations can result in large standard errors of the OLS estimates due to the X`X matrix being ill-conditioned. You can use the VIF option in PROC REG to examine whether the X`X matrix is ill-conditioned.

If the VIF indicates strong correlations, you might try ridged regression in PROC REG, which is close to the Elastic Net in that it includes the quadratic penalty term. That would probably permit the closest comparison.

For other options for regression of correlated variables, see a comparison of PLS and PCR (principal component regression).

Re: proc gmlselect + elastic net without feature selection

csetzkorn — Mon, 23 Apr 2018 10:24:57 GMT

Thanks. Yes ridge is an option.

Re: proc gmlselect + elastic net without feature selection

PaigeMiller — Mon, 23 Apr 2018 13:24:25 GMT

I would say that PLS is superior to PCR in the case of correlated X variables, because PCR may find components that are not good predictors of the Y variable(s), while PLS will find components of X that predict Y well (if such components exist). Studies show that PLS does well with correlated variables, compared to other methods such as stepwise regression, ridge regression and even PCR.

https://amstat.tandfonline.com/doi/abs/10.1080/00401706.1993.10485033