Programming the statistical procedures from SAS

Finding Best Linear Regression Model using AIC

Reply
New Contributor
Posts: 3

Finding Best Linear Regression Model using AIC

Any help is much appreciated!

 

I was required to run a code for class with a provided dataset to find the 5 best linear regression models in terms of AIC. This is my output:

3a.PNG

 

I understand the top 5 are my 5 best models, but there is not an Intercept listed. Is this possible to not have a y-intercept in the model? I was trying to read up on this, but there seems to be some conflicting opinions.

 

Thank you in advance for the help!

 

Dan

Respected Advisor
Posts: 3,040

Re: Finding Best Linear Regression Model using AIC

Yes, it is possible to have no intercept, effectively it means that when all x variables = 0, that y=0.

 

Of course, the next question to ask is: is it a good idea to fit models with no intercepts. My answer is that I am usually skeptical when someone fits a model with no intercept, without strong justification for doing so.

--
Paige Miller
New Contributor
Posts: 3

Re: Finding Best Linear Regression Model using AIC

Posted in reply to PaigeMiller

Thank you!

 

So when I used AIC to find the "best 5 models", it's telling me the models that are the most "significant" are those that don't have a y-intercept? Could that mean that the y-intercept may not be very statistically significant itself and therefore the "best models" don't include it?

 

Thanks!

New Contributor
Posts: 3

Re: Finding Best Linear Regression Model using AIC

Also, when using Backward Elimination Method, Forward Selection Method, or Stepwise Selection method this was the "best" model:

 

3b.PNG

 

All three of these methods were the same with a y-intercept (intercept, x1, x6, x9). The best method for AIC was the same except for no intercept (x1,x6,x9).

 

This is what's confusing me; not sure why the intercept is left out for the models in AIC....

Respected Advisor
Posts: 3,040

Re: Finding Best Linear Regression Model using AIC

Since you don't show us your code, we don't know why there is not intercept in your models; all I can say is that I think it is a very poor idea to leave out the intercept without strong justification.

when I used AIC to find the "best 5 models", it's telling me the models that are the most "significant" are those that don't have a y-intercept? Could that mean that the y-intercept may not be very statistically significant itself and therefore the "best models" don't include it?

 

No, I don't think that's what it is saying, at least using the statistical meaning of "significant". It means that using AIC, SAS chose models that did not have the intercept as a model term. AIC and "significance" in the statistical sense are not the same thing.

 

I am also usually opposed to any form of stepwise regression, as you can read on the internet dozens of people writing about dozens of drawbacks regarding this method.

--
Paige Miller
Super User
Posts: 10,784

Re: Finding Best Linear Regression Model using AIC

I would suggest to use PROC PLS to pick up the significant variables ,which would not become over-fit model .

Check the example of PROC PLS in documentation.

Respected Advisor
Posts: 3,040

Re: Finding Best Linear Regression Model using AIC


Ksharp wrote:

I would suggest to use PROC PLS to pick up the significant variables ,which would not become over-fit model .

Check the example of PROC PLS in documentation.


I wish I had said that ... Smiley Tongue

--
Paige Miller
Ask a Question
Discussion stats
  • 6 replies
  • 126 views
  • 1 like
  • 3 in conversation