topic F Value in ANOVA -- The bigger the better? in SAS Procedures

F Value in ANOVA -- The bigger the better?

NKormanik — Tue, 26 Mar 2019 10:37:29 GMT

Suppose two ANOVA models, #1 and #2.

In #1 the F value is 500.

In #2 the F value is 50.

Same degrees of freedom.

Both, of course, are highly significant -- p<.0001.

Can we say anything about the 500 vs 50?

Is model #1 in some way better than #2?

Thanks!

Nicholas Kormanik

Re: F Value in ANOVA -- The bigger the better?

PaigeMiller — Tue, 26 Mar 2019 10:47:40 GMT

@NKormanik wrote:

Suppose two ANOVA models, #1 and #2.

In #1 the F value is 500.

In #2 the F value is 50.

Same degrees of freedom.

Both, of course, are highly significant -- p<.0001.

Can we say anything about the 500 vs 50?

It is true that if you have more data points (i.e. more degrees of freedom), you will have a better chance to achieve statistical significance with the same level of error.

Is model #1 in some way better than #2?

Define "better".

Re: F Value in ANOVA -- The bigger the better?

NKormanik — Tue, 26 Mar 2019 12:05:22 GMT

"Better" means..., if you were going to place a bet. $1000. Based on one
model versus the other.

On which model would you place your bet?

Both models are significant, as said above.

Re: F Value in ANOVA -- The bigger the better?

PaigeMiller — Tue, 26 Mar 2019 12:11:44 GMT

I don't gamble, first. So I decline to answer.

Second, I have no idea what these models are or what they predict, I don't have your data, and you haven't provided the measures of goodness-of-fit, so I decline to answer for that reason as well.

Re: F Value in ANOVA -- The bigger the better?

Ksharp — Tue, 26 Mar 2019 12:20:38 GMT

I think so.

model #1 in some way better than #2

Re: F Value in ANOVA -- The bigger the better?

PaigeMiller — Tue, 26 Mar 2019 12:26:49 GMT

@Ksharp wrote:

I think so.

model #1 in some way better than #2

Ok, I'll have to disagree with this, unless you are using a different meaning of "better" than I am. I want to see goodness of fit statistics, and not F-test, to determine which model fits better. Without goodness of fit measures, I can't say which model is better. If we're not talking about how well the model fits, then what definition of "better" are you using?

Re: F Value in ANOVA -- The bigger the better?

NKormanik — Thu, 28 Mar 2019 01:08:23 GMT

Same true of t-values?

Say, two t-values are significant, but one is 10X larger than another.

Could we say that the variable with the much larger t-value is in some way more significant?

Does seem reasonable.

Re: F Value in ANOVA -- The bigger the better?

NKormanik — Thu, 28 Mar 2019 05:40:00 GMT

What I've asked here seems like an entirely reasonable question:

Comparing hugely different F-values (and, t-values).

What can one (a very experienced expert statistician) say regarding such differences.

In researching, however, I came across a series of articles that seem to question the foundations, and pop the balloon:

https://www.tandfonline.com/toc/utas20/current

One example:

Moving to a World Beyond “p < 0.05”

Good grief.

Re: F Value in ANOVA -- The bigger the better?

Ksharp — Thu, 28 Mar 2019 11:49:40 GMT

Yeah. I think so.

t-value and F-value in some way are the same thing, they all measure the deviation under H0 is true .

So bigger is more significant .

But You should listen to Paige's advice . Also Check Goodness of Fitness test .

Re: F Value in ANOVA -- The bigger the better?

PaigeMiller — Thu, 28 Mar 2019 12:00:39 GMT

Goodness of fit measures tell you how well a model fits; not F-tests, not t-tests.

Re: F Value in ANOVA -- The bigger the better?

NKormanik — Sat, 30 Mar 2019 11:31:46 GMT

Would the following be reasonable candidates for "Goodness of Fit"? I see they are provided through Proc Mixed, which I was using.

Fit Statistics

-2 Res Log Likelihood

AIC (Smaller is Better)

AICC (Smaller is Better)

BIC (Smaller is Better)

Thanks!

Re: F Value in ANOVA -- The bigger the better?

Ksharp — Sat, 30 Mar 2019 13:16:48 GMT

Yes. That is right way to compare two model .