topic Re: how to check the Multicollinearity between continuous and categorical variables? in Statistical Procedures

how to check the Multicollinearity between continuous and categorical variables?

joe66 — Mon, 09 Apr 2018 16:00:15 GMT

Hi,

I am running a logistic model that includes continuous and categorical variables, should I still need to check Multicollinearity between them? And how to do that?

I know I can ingore the correlation test since they are not all continuous variables, but I am not sure how to check Multicollinearity for these two kinds of variables in logistic regression.

Thanks,

Joe

Re: how to check the Multicollinearity between continuous and categorical variables?

PaigeMiller — Mon, 09 Apr 2018 17:25:01 GMT

I think it is a very safe assumption that you have some (or maybe a lot of) multi-collinearity, so what are you going to do in the presence of multi-collinearity?

Logistic regression does not perform well in the presence of multi-collinearity; a better method, in my opinion is to use a method that performs well in the presence of multi-collinearity; that method is Partial Least Squares Regression. Unfortunately, there is no logistic Partial Least SQuares in SAS, however the method has been published at https://cedric.cnam.fr/fichiers/RC906.pdf

Re: how to check the Multicollinearity between continuous and categorical variables?

joe66 — Mon, 09 Apr 2018 18:57:49 GMT

Thank you for the comments!

I also think it is necessary to check the Multicollinearity among these continuous and categorical predictors, I do not find the "VIF" option in Proc logistic statement, so is there any other way I can check the Multicollinearity?

Re: how to check the Multicollinearity between continuous and categorical variables?

PaigeMiller — Mon, 09 Apr 2018 21:24:28 GMT

Again, if you just assume there is multi-collinearity, what have you lost by not actually checking? Almost all real world data has multi-collinearity. What are you going to do about multi-collinearity if it is present?

Re: how to check the Multicollinearity between continuous and categorical variables?

PaigeMiller — Mon, 09 Apr 2018 21:36:07 GMT

Adding, if you really want to check the VIF, you can do that in PROC REG, the calculation of VIF is correct for both ordinary least squares regression and for logistic regression, it doesn't matter.

Re: how to check the Multicollinearity between continuous and categorical variables?

Ksharp — Tue, 10 Apr 2018 13:56:17 GMT

You could try CORRB option.

proc logistic data=sashelp.class;
model sex=age weight height/corrb;
run;

You will get correlation matrix of coefficient.

For example, Here HEIGHT and INTERCEPT has -0.82 correlation coefficient. therefore they are correlation.

Re: how to check the Multicollinearity between continuous and categorical variables?

StatDave — Tue, 10 Apr 2018 14:04:32 GMT

See this note about assessing collinearity in generalized linear models.

Re: how to check the Multicollinearity between continuous and categorical variables?

PaigeMiller — Tue, 10 Apr 2018 14:18:15 GMT

@Ksharp wrote:

You could try CORRB option.
proc logistic data=sashelp.class;
model sex=age weight height/corrb;
run;
You will get correlation matrix of coefficient.

For example, Here HEIGHT and INTERCEPT has -0.82 correlation coefficient. therefore they are correlation.

This is not the same as having correlation between the original variables.

As stated in the link given by @StatDave, "Extremely large standard errors for one or more of the estimated parameters and large off-diagonal values in the parameter covariance matrix (COVB option) or correlation matrix (CORRB option) both suggest an ill-conditioned information matrix. However, these conditions can also happen for reasons not related to collinearity among the raw predictors."

But all this still seems to ignore the question: You almost certainly have collinearity among the predictors; all real world data does. So, what are you going to do about collinearity?

Re: how to check the Multicollinearity between continuous and categorical variables?

joe66 — Tue, 10 Apr 2018 16:10:24 GMT

Hi Paige,

Thanks for your answer. I am using VIF to deal with multicollinerarity, remove the variable from my model if VIF>10, keep it otherwise. Did I understand your question correctly?