Lasso Logistics regression

[AmitKB]

Hi all,
Is their a way to implement Lasso Logistics Regression. I am looking to use it for variable selection.
Thank for all your help.

Regards,

Amit

[plf515]

> Hi all,
> Is their a way to implement Lasso Logistics
> Regression. I am looking to use it for variable
> selection.
> Thank for all your help.
> ,
>
> Amit

Hi Amit

You can do this in two steps:
1) Use GLMSELECT as if you had an OLS model, and get several sensible models
2) Try those models in LOGISTIC or whichever PROC you like for logistic regression.

This isn't formally "right" but I've had good success with it.

There are also some papers on lasso for logistic models; see my paper with David Cassell on Stopping Stepwise

http://www.nesug.org/Proceedings/nesug09/sa/sa01.pdf

HTH

Peter

[AmitKB]

Hi Peter,
I appreciate all your help. Thanks for replying to my question.

There are so many options in the proc glmselect for selection=lasso, I am lost. In your experience, what are some of the options that have worked for binary response variable.

If you could give me some sample code that would be great.

Regards,

Amit

[yonggang]

You may try PROC QUANTSELECT to perform Support Vector Machine (SVM) classification with LASSO penalty. An example is as follows:

/* Start: data simulation */
%let seed=111;
Data raw;
     array x[10];
     do i=1 to 1000;
          x0=1; /* regressor for estimating bias parameter */
          do j=1 to 10;
              x = rannor(&seed);
          end;
          y = 2*((3*x1+2*x2+x3+1+0.1*rannor(&seed))>0)-1; /*So the true model is proportional to (bias=1, x1=3, x2=2, x1=1).*/
          output;
     end;
run;
/* End: data simulation */

/* TransformSVM2QR macro pre-processes raw data for using QUANTSELECT.*/
%macro TransformSVM2QR(raw);
data transferred_&raw;
   set &raw;
   %do j=1 %to 10;
       x&j = y*x&j;
   %end;
   bias=y;
   y=1;
   run;
%mend TransformSVM2QR;

%TransformSVM2QR(raw);

ods graphics on;

proc quantselect data=transferred_raw plot=all;
   model y= bias x1-x10/quantile=1 noint selection=lasso(choose=validate include=1 sh=3);
   partition fraction(validate=0.2);
run;