topic Re: ERROR: Model selection by lassos could not be completed in SAS Procedures

ERROR: Model selection by lassos could not be completed

BTAinRVA — Wed, 03 Feb 2016 16:27:25 GMT

Proc HPGENSELECT and the LASSO method are new to me and i wanted to see it in action with some real data. Unfortunately when I run the code below I the the error message above in the log but I get some output indicating that the intercept only model was selected as the best model.

proc hpgenselect data=SSD_DeIDd;

model SS_vis_ct= c_MH_status c_gender c_age p_age p_gender cumul_length_of_deploy /dist=nb link=log ;

selection method=Lasso(choose=SBC) details=all;

performance details;

run;

I did a google search and got nothing.

Thanks!

Re: ERROR: Model selection by lassos could not be completed

BTAinRVA — Wed, 03 Feb 2016 17:56:39 GMT

P.S. When I analyze the same data using Proc Genmod it finds all but one variable significant using either forward, backward, or stepwise methods.

Re: ERROR: Model selection by lassos could not be completed

Rick_SAS — Wed, 03 Feb 2016 19:44:49 GMT

It is very difficult to diagnose a "lack of convergence" issue when there is no data. However, the documentation for the LASSO selection option includes the statement

Some distributions involve a dispersion parameter (the parameter in the expressions for the log likelihood)..... These parameters are not estimated by the LASSO optimization algorithm, and are set to either the default value or a value that you specify. You can use the MODEL statement options PHI= to set the dispersion to a fixed value.

I don't know if it will help, but I'd try plugging in the value of the dispersion parameter from the GENMOD run. Can't hurt, and you might get lucky.

Re: ERROR: Model selection by lassos could not be completed

BTAinRVA — Thu, 04 Feb 2016 20:59:58 GMT

Rick,

Thanks for the reply. I tried using the PHI option but got the same result, only the intercept model as the selected model and the error message as shown in the subject line.

Any ideas why the HPGENSELECT seems to work with my data when I use selection method FORWARD, BACKWARD and STEPWISE, but it doesn't seem to work with LASSO?

Thanks again!

Re: ERROR: Model selection by lassos could not be completed

Rick_SAS — Thu, 04 Feb 2016 21:14:16 GMT

No, I can't guess based on what you've told us. LASSO performs a nonlinear optimization, so it could be any of a dozen reasons. The fact that other variable selection techniques converge isn't relevant or informative.

Re: ERROR: Model selection by lassos could not be completed

BTAinRVA — Thu, 04 Feb 2016 21:35:05 GMT

Thanks for the quick reply! What other information would be helpful?

Re: ERROR: Model selection by lassos could not be completed

Rick_SAS — Thu, 04 Feb 2016 21:56:19 GMT

The usual: The data and program that shows the problem.

Re: ERROR: Model selection by lassos could not be completed

BTAinRVA — Thu, 04 Feb 2016 22:24:57 GMT

Unfortunately I can't share the data. What we are trying to do is model count of child mental health visits against cumulative parental deployment, child age, child gender, child mental health status, parent age, parent gender, parent rank. There are over 8800 records. When I run the code below with either Forward, Backward, or Stepwise method, all factors but one are selected for the final model. When I run the code with the Lasso method I just get the intercept model and the error in the log.

proc hpgenselect data=SSD_DeIDd;

class adol_gender adol_MH_status spon_gender spon_rank;

model adol_SS_vis_ct= cumul_length_of_deploy adol_age adol_gender adol_MH_status spon_gender spon_age spon_rank / dist=nb link=log offset=ln_obs_time ;

selection method=forward (choose=SBC)details=all;

/* selection method=Lasso(choose=SBC) details=all;*/

run;