PROC LOGISTIC variable selection supress linear combination

[stratozyck]

Hi, we are trying to do variable selection and we keep getting "Note: The following variables are not used in the SCORE selection since they are a linear combination of other variables as shown."

Is there a way to make all variables used? We need to have all variables used in the selection option of proc logistic even if there are linear combinations.

[Reeza]

Please post your full code and log.

[PaigeMiller]

---

@stratozyck wrote:

Hi, we are trying to do variable selection and we keep getting "Note: The following variables are not used in the SCORE selection since they are a linear combination of other variables as shown."

Is there a way to make all variables used? We need to have all variables used in the selection option of proc logistic even if there are linear combinations.

---

No, that's the problem when some variables are linear combinations of other variables. They cannot all be used by PROC LOGISTIC. And it doesn't make sense to use all of them anyway.

[stratozyck]

The code is:

ODS OUTPUT Bestsubsets = gridwork.qry_subsets&seg;
proc logistic data=&indata;
model outcome(EVENT='1')= &list
/selection=score best= &modnum start=&minnum stop=&varnum;
run;

What we want it to do is consider all variables in sets of 3-4 or so. Lets say we have x1-x100. We want it to consider all of them in sets of 3-4. 

I get what you are saying about the variables not being used in the same model, but if x2 is a linear combination of x5 and x6, we still want x2 considered even if x5 and x6 are not in the model.

[Reeza]

You run into problems manipulating the matrices when it's an exact linear combination, it's not a matter of 'allowing' something to occur.

[stratozyck]

No there would be no problem. Lets say X1 is 2*X2

Obviouslt including both X1 and X2 is an issue but the selection procedure only considers one and doesnt consider the other in any way.

Its more complex than that. It dops a variable from consideration because its a linear combo of like 10 other variables even though we told it to only consider 3 per candidate model.

Again, we are NOT trying to put variables that are linear combinations into one model, but want it to consider variables it says are linear combinations in seperate candiate models.

[Reeza]

---

@stratozyck wrote:
No there would be no problem. Lets say X1 is 2*X2

Obviouslt including both X1 and X2 is an issue but the selection procedure only considers one and doesnt consider the other in any way.

Its more complex than that. It dops a variable from consideration because its a linear combo of like 10 other variables even though we told it to only consider 3 per candidate model.

Again, we are NOT trying to put variables that are linear combinations into one model, but want it to consider variables it says are linear combinations in seperate candiate models.

---

It'll run fine within an individual process, but I'm not seeing how you're controlling for 3 per candidate model. I'm assuming that's some sort of macro but the code shown is just the logistic regression. You could a KEEP statement to each iteration to ensure it's just the variables you want considered.

[PaigeMiller]

---

@stratozyck wrote:
No there would be no problem. Lets say X1 is 2*X2

Obviouslt including both X1 and X2 is an issue but the selection procedure only considers one and doesnt consider the other in any way.

Its more complex than that. It dops a variable from consideration because its a linear combo of like 10 other variables even though we told it to only consider 3 per candidate model.

Again, we are NOT trying to put variables that are linear combinations into one model, but want it to consider variables it says are linear combinations in seperate candiate models.

---

The restriction isn't because of SAS, it comes from the fact that the math doesn't allow it. Any matrix that has linear combinations of the X-variables cannot be inverted and so cannot be used in logistic (or ordinary least squares) regression.

Also

What we want it to do is consider all variables in sets of 3-4 or so. Lets say we have x1-x100. We want it to consider all of them in sets of 3-4. 

I have no idea whether this is a good idea or not, but in my mind, I wouldn't do this when you have 100 x-variables. The collinearity between this 100 x-variables is going to cause problems with whatever selection method is used.

Perhaps, given the explanation above, that you have 100 correlated input variables, some of them a linear combination of other variables, you need a change of mindset.

You need a method that is better at handling correlated input variables than either logistic regression or ordinary least squares regression. You also need a method that can handle situations where the input matrix has linear combinations of your x-variables. That method is Partial Least Squares regression, but it requires a change of mindset, where now you KEEP all of your x-variables in the model, many of them will have weightings (loadings is the technical term) that are very close to zero. Partial Least Squares is much better than ordinary least squares regression or logistic regression at modelling with lots of input variables; and it also can work when you have linear combinations of input variables.

[Ksharp]

Check  include= option of model statement.

[PaigeMiller]

Just because you can use the INCLUDE option doesn't mean you should use the INCLUDE option.