## Lasso Logistics regression

Frequent Contributor
Posts: 99

# Lasso Logistics regression

Hi all,
Is their a way to implement Lasso Logistics Regression. I am looking to use it for variable selection.

Regards,

Amit
Frequent Contributor
Posts: 140

## Re: Lasso Logistics regression

> Hi all,
> Is their a way to implement Lasso Logistics
> Regression. I am looking to use it for variable
> selection.
> Thank for all your help.
> ,
>
> Amit

Hi Amit

You can do this in two steps:
1) Use GLMSELECT as if you had an OLS model, and get several sensible models
2) Try those models in LOGISTIC or whichever PROC you like for logistic regression.

This isn't formally "right" but I've had good success with it.

There are also some papers on lasso for logistic models; see my paper with David Cassell on Stopping Stepwise

http://www.nesug.org/Proceedings/nesug09/sa/sa01.pdf

HTH

Peter
Frequent Contributor
Posts: 99

## Re: Lasso Logistics regression

Hi Peter,

There are so many options in the proc glmselect for selection=lasso, I am lost. In your experience, what are some of the options that have worked for binary response variable.

If you could give me some sample code that would be great.

Regards,

Amit
SAS Employee
Posts: 6

## Re: Lasso Logistics regression

You may try PROC QUANTSELECT to perform Support Vector Machine (SVM) classification with LASSO penalty. An example is as follows:

/* Start: data simulation */
%let seed=111;
Data raw;
array x[10];
do i=1 to 1000;
x0=1; /* regressor for estimating bias parameter */
do j=1 to 10;
x = rannor(&seed);
end;
y = 2*((3*x1+2*x2+x3+1+0.1*rannor(&seed))>0)-1; /*So the true model is proportional to (bias=1, x1=3, x2=2, x1=1).*/
output;
end;
run;
/* End: data simulation */

/* TransformSVM2QR macro pre-processes raw data for using QUANTSELECT.*/
%macro TransformSVM2QR(raw);
data transferred_&raw;
set &raw;
%do j=1 %to 10;
x&j = y*x&j;
%end;
bias=y;
y=1;
run;
%mend TransformSVM2QR;

%TransformSVM2QR(raw);

ods graphics on;

proc quantselect data=transferred_raw plot=all;
model y= bias x1-x10/quantile=1 noint selection=lasso(choose=validate include=1 sh=3);
partition fraction(validate=0.2);
run;

Discussion stats
• 3 replies
• 2746 views
• 0 likes
• 3 in conversation