topic Re: Lasso Logistics regression in Statistical Procedures

Lasso Logistics regression

AmitKB — Thu, 18 Feb 2010 17:36:14 GMT

Hi all,
Is their a way to implement Lasso Logistics Regression. I am looking to use it for variable selection.
Thank for all your help.

Regards,

Amit

Re: Lasso Logistics regression

plf515 — Fri, 19 Feb 2010 01:50:30 GMT

> Hi all,
> Is their a way to implement Lasso Logistics
> Regression. I am looking to use it for variable
> selection.
> Thank for all your help.
> ,
>
> Amit

Hi Amit

You can do this in two steps:
1) Use GLMSELECT as if you had an OLS model, and get several sensible models
2) Try those models in LOGISTIC or whichever PROC you like for logistic regression.

This isn't formally "right" but I've had good success with it.

There are also some papers on lasso for logistic models; see my paper with David Cassell on Stopping Stepwise

http://www.nesug.org/Proceedings/nesug09/sa/sa01.pdf

HTH

Peter

Re: Lasso Logistics regression

AmitKB — Fri, 19 Feb 2010 15:22:52 GMT

Hi Peter,
I appreciate all your help. Thanks for replying to my question.

There are so many options in the proc glmselect for selection=lasso, I am lost. In your experience, what are some of the options that have worked for binary response variable.

If you could give me some sample code that would be great.

Regards,

Amit

Re: Lasso Logistics regression

yonggang — Wed, 16 Oct 2013 03:26:48 GMT

You may try PROC QUANTSELECT to perform Support Vector Machine (SVM) classification with LASSO penalty. An example is as follows:

/* Start: data simulation */
%let seed=111;
Data raw;
     array x[10];
     do i=1 to 1000;
          x0=1; /* regressor for estimating bias parameter */
          do j=1 to 10;
              x = rannor(&seed);
          end;
          y = 2*((3*x1+2*x2+x3+1+0.1*rannor(&seed))>0)-1; /*So the true model is proportional to (bias=1, x1=3, x2=2, x1=1).*/
          output;
     end;
run;
/* End: data simulation */

/* TransformSVM2QR macro pre-processes raw data for using QUANTSELECT.*/
%macro TransformSVM2QR(raw);
data transferred_&raw;
   set &raw;
   %do j=1 %to 10;
       x&j = y*x&j;
   %end;
   bias=y;
   y=1;
   run;
%mend TransformSVM2QR;

%TransformSVM2QR(raw);

ods graphics on;

proc quantselect data=transferred_raw plot=all;
model y= bias x1-x10/quantile=1 noint selection=lasso(choose=validate include=1 sh=3);
partition fraction(validate=0.2);
run;