topic Re: Finding Best Linear Regression Model using AIC in Statistical Procedures

Finding Best Linear Regression Model using AIC

DKotch22 — Sat, 03 Feb 2018 15:34:09 GMT

Any help is much appreciated!

I was required to run a code for class with a provided dataset to find the 5 best linear regression models in terms of AIC. This is my output:

I understand the top 5 are my 5 best models, but there is not an Intercept listed. Is this possible to not have a y-intercept in the model? I was trying to read up on this, but there seems to be some conflicting opinions.

Thank you in advance for the help!

Dan

Re: Finding Best Linear Regression Model using AIC

PaigeMiller — Sat, 03 Feb 2018 16:04:42 GMT

Yes, it is possible to have no intercept, effectively it means that when all x variables = 0, that y=0.

Of course, the next question to ask is: is it a good idea to fit models with no intercepts. My answer is that I am usually skeptical when someone fits a model with no intercept, without strong justification for doing so.

Re: Finding Best Linear Regression Model using AIC

DKotch22 — Sat, 03 Feb 2018 16:23:00 GMT

Thank you!

So when I used AIC to find the "best 5 models", it's telling me the models that are the most "significant" are those that don't have a y-intercept? Could that mean that the y-intercept may not be very statistically significant itself and therefore the "best models" don't include it?

Thanks!

Re: Finding Best Linear Regression Model using AIC

DKotch22 — Sat, 03 Feb 2018 16:47:11 GMT

Also, when using Backward Elimination Method, Forward Selection Method, or Stepwise Selection method this was the "best" model:

All three of these methods were the same with a y-intercept (intercept, x1, x6, x9). The best method for AIC was the same except for no intercept (x1,x6,x9).

This is what's confusing me; not sure why the intercept is left out for the models in AIC....

Re: Finding Best Linear Regression Model using AIC

PaigeMiller — Sat, 03 Feb 2018 17:29:37 GMT

Since you don't show us your code, we don't know why there is not intercept in your models; all I can say is that I think it is a very poor idea to leave out the intercept without strong justification.

when I used AIC to find the "best 5 models", it's telling me the models that are the most "significant" are those that don't have a y-intercept? Could that mean that the y-intercept may not be very statistically significant itself and therefore the "best models" don't include it?

No, I don't think that's what it is saying, at least using the statistical meaning of "significant". It means that using AIC, SAS chose models that did not have the intercept as a model term. AIC and "significance" in the statistical sense are not the same thing.

I am also usually opposed to any form of stepwise regression, as you can read on the internet dozens of people writing about dozens of drawbacks regarding this method.

Re: Finding Best Linear Regression Model using AIC

Ksharp — Sun, 04 Feb 2018 10:31:18 GMT

I would suggest to use PROC PLS to pick up the significant variables ,which would not become over-fit model .

Check the example of PROC PLS in documentation.

Re: Finding Best Linear Regression Model using AIC

PaigeMiller — Sun, 04 Feb 2018 12:46:53 GMT

@Ksharp wrote:

I would suggest to use PROC PLS to pick up the significant variables ,which would not become over-fit model .

Check the example of PROC PLS in documentation.

I wish I had said that ...