F Value in ANOVA -- The bigger the better?

[NKormanik]

Suppose two ANOVA models, #1 and #2.

In #1 the F value is 500.

In #2 the F value is 50.

Same degrees of freedom.

Both, of course, are highly significant -- p<.0001.

Can we say anything about the 500 vs 50?

Is model #1 in some way better than #2?

Thanks!

Nicholas Kormanik

[PaigeMiller]

---

@NKormanik wrote:

Suppose two ANOVA models, #1 and #2.

In #1 the F value is 500.

In #2 the F value is 50.

Same degrees of freedom.

Both, of course, are highly significant -- p<.0001.

Can we say anything about the 500 vs 50?

---

It is true that if you have more data points (i.e. more degrees of freedom), you will have a better chance to achieve statistical significance with the same level of error.

Is model #1 in some way better than #2?

Define "better".

[NKormanik]

"Better" means..., if you were going to place a bet. $1000. Based on one
model versus the other.

On which model would you place your bet?

Both models are significant, as said above.

[PaigeMiller]

I don't gamble, first. So I decline to answer.

Second, I have no idea what these models are or what they predict, I don't have your data, and you haven't provided the measures of goodness-of-fit, so I decline to answer for that reason as well.

[Ksharp · Accepted solution]

I think so.

 model #1 in some way better than #2

[PaigeMiller]

---

@Ksharp wrote:

I think so.

 model #1 in some way better than #2

---

Ok, I'll have to disagree with this, unless you are using a different meaning of "better" than I am. I want to see goodness of fit statistics, and not F-test, to determine which model fits better. Without goodness of fit measures, I can't say which model is better. If we're not talking about how well the model fits, then what definition of "better" are you using? 

[NKormanik]

Same true of t-values?

Say, two t-values are significant, but one is 10X larger than another.

Could we say that the variable with the much larger t-value is in some way more significant?

Does seem reasonable.

[NKormanik]

What I've asked here seems like an entirely reasonable question:

Comparing hugely different F-values (and, t-values).

What can one (a very experienced expert statistician) say regarding such differences.

In researching, however, I came across a series of articles that seem to question the foundations, and pop the balloon:

https://www.tandfonline.com/toc/utas20/current

One example:

Moving to a World Beyond “p < 0.05”

Good grief.

[PaigeMiller]

Goodness of fit measures tell you how well a model fits; not F-tests, not t-tests.

[Ksharp]

Yeah. I think so.

t-value and F-value in some way are the same thing, they all measure the deviation under H0 is true .

So bigger is more significant .

But You should listen to Paige's advice . Also Check Goodness of Fitness test .

[NKormanik]

Would the following be reasonable candidates for "Goodness of Fit"?  I see they are provided through Proc Mixed, which I was using.

Fit Statistics

-2 Res Log Likelihood

AIC (Smaller is Better)

AICC (Smaller is Better)

BIC (Smaller is Better)

Thanks!

[Ksharp]

Yes. That is right way to compare two model .