topic Re: PROC LOGISTIC variable selection supress linear combination in Statistical Procedures

PROC LOGISTIC variable selection supress linear combination

stratozyck — Fri, 03 Nov 2017 14:55:47 GMT

Hi, we are trying to do variable selection and we keep getting "Note: The following variables are not used in the SCORE selection since they are a linear combination of other variables as shown."

Is there a way to make all variables used? We need to have all variables used in the selection option of proc logistic even if there are linear combinations.

Re: PROC LOGISTIC variable selection supress linear combination

Reeza — Fri, 03 Nov 2017 15:02:14 GMT

Please post your full code and log.

Re: PROC LOGISTIC variable selection supress linear combination

PaigeMiller — Fri, 03 Nov 2017 15:21:18 GMT

@stratozyck wrote:

Hi, we are trying to do variable selection and we keep getting "Note: The following variables are not used in the SCORE selection since they are a linear combination of other variables as shown."

Is there a way to make all variables used? We need to have all variables used in the selection option of proc logistic even if there are linear combinations.

No, that's the problem when some variables are linear combinations of other variables. They cannot all be used by PROC LOGISTIC. And it doesn't make sense to use all of them anyway.

Re: PROC LOGISTIC variable selection supress linear combination

stratozyck — Fri, 03 Nov 2017 15:48:21 GMT

The code is:

ODS OUTPUT Bestsubsets = gridwork.qry_subsets&seg;
proc logistic data=&indata;
model outcome(EVENT='1')= &list
/selection=score best= &modnum start=&minnum stop=&varnum;
run;

What we want it to do is consider all variables in sets of 3-4 or so. Lets say we have x1-x100. We want it to consider all of them in sets of 3-4.

I get what you are saying about the variables not being used in the same model, but if x2 is a linear combination of x5 and x6, we still want x2 considered even if x5 and x6 are not in the model.

Re: PROC LOGISTIC variable selection supress linear combination

Reeza — Fri, 03 Nov 2017 16:24:10 GMT

You run into problems manipulating the matrices when it's an exact linear combination, it's not a matter of 'allowing' something to occur.

Re: PROC LOGISTIC variable selection supress linear combination

stratozyck — Fri, 03 Nov 2017 16:33:11 GMT

No there would be no problem. Lets say X1 is 2*X2

Obviouslt including both X1 and X2 is an issue but the selection procedure only considers one and doesnt consider the other in any way.

Its more complex than that. It dops a variable from consideration because its a linear combo of like 10 other variables even though we told it to only consider 3 per candidate model.

Again, we are NOT trying to put variables that are linear combinations into one model, but want it to consider variables it says are linear combinations in seperate candiate models.

Re: PROC LOGISTIC variable selection supress linear combination

Reeza — Fri, 03 Nov 2017 16:42:12 GMT

@stratozyck wrote:
No there would be no problem. Lets say X1 is 2*X2

Obviouslt including both X1 and X2 is an issue but the selection procedure only considers one and doesnt consider the other in any way.

Its more complex than that. It dops a variable from consideration because its a linear combo of like 10 other variables even though we told it to only consider 3 per candidate model.

Again, we are NOT trying to put variables that are linear combinations into one model, but want it to consider variables it says are linear combinations in seperate candiate models.

It'll run fine within an individual process, but I'm not seeing how you're controlling for 3 per candidate model. I'm assuming that's some sort of macro but the code shown is just the logistic regression. You could a KEEP statement to each iteration to ensure it's just the variables you want considered.

Re: PROC LOGISTIC variable selection supress linear combination

PaigeMiller — Sat, 04 Nov 2017 10:37:11 GMT

@stratozyck wrote:
No there would be no problem. Lets say X1 is 2*X2

Obviouslt including both X1 and X2 is an issue but the selection procedure only considers one and doesnt consider the other in any way.

Its more complex than that. It dops a variable from consideration because its a linear combo of like 10 other variables even though we told it to only consider 3 per candidate model.

Again, we are NOT trying to put variables that are linear combinations into one model, but want it to consider variables it says are linear combinations in seperate candiate models.

The restriction isn't because of SAS, it comes from the fact that the math doesn't allow it. Any matrix that has linear combinations of the X-variables cannot be inverted and so cannot be used in logistic (or ordinary least squares) regression.

Also

What we want it to do is consider all variables in sets of 3-4 or so. Lets say we have x1-x100. We want it to consider all of them in sets of 3-4.

I have no idea whether this is a good idea or not, but in my mind, I wouldn't do this when you have 100 x-variables. The collinearity between this 100 x-variables is going to cause problems with whatever selection method is used.

Perhaps, given the explanation above, that you have 100 correlated input variables, some of them a linear combination of other variables, you need a change of mindset.

You need a method that is better at handling correlated input variables than either logistic regression or ordinary least squares regression. You also need a method that can handle situations where the input matrix has linear combinations of your x-variables. That method is Partial Least Squares regression, but it requires a change of mindset, where now you KEEP all of your x-variables in the model, many of them will have weightings (loadings is the technical term) that are very close to zero. Partial Least Squares is much better than ordinary least squares regression or logistic regression at modelling with lots of input variables; and it also can work when you have linear combinations of input variables.

Re: PROC LOGISTIC variable selection supress linear combination

Ksharp — Sat, 04 Nov 2017 13:00:31 GMT

Check include= option of model statement.

Re: PROC LOGISTIC variable selection supress linear combination

PaigeMiller — Sat, 04 Nov 2017 17:10:44 GMT

Just because you can use the INCLUDE option doesn't mean you should use the INCLUDE option.