SAS Procedures

Help using Base SAS procedures
BookmarkSubscribeRSS Feed
🔒 This topic is solved and locked. Need further help from the community? Please sign in and ask a new question.
NKormanik
Barite | Level 11

Suppose two ANOVA models, #1 and #2.

 

In #1 the F value is 500.

 

In #2 the F value is 50.

 

Same degrees of freedom.

 

Both, of course, are highly significant -- p<.0001.

 

Can we say anything about the 500 vs 50?

 

Is model #1 in some way better than #2?

 

Thanks!

 

Nicholas Kormanik

 

 

1 ACCEPTED SOLUTION

Accepted Solutions
Ksharp
Super User

I think so.

 model #1 in some way better than #2

View solution in original post

11 REPLIES 11
PaigeMiller
Diamond | Level 26

@NKormanik wrote:

Suppose two ANOVA models, #1 and #2.

 

In #1 the F value is 500.

 

In #2 the F value is 50.

 

Same degrees of freedom.

 

Both, of course, are highly significant -- p<.0001.

 

Can we say anything about the 500 vs 50?

 


It is true that if you have more data points (i.e. more degrees of freedom), you will have a better chance to achieve statistical significance with the same level of error.

 

Is model #1 in some way better than #2?

 

Define "better".

--
Paige Miller
NKormanik
Barite | Level 11
"Better" means..., if you were going to place a bet. $1000. Based on one
model versus the other.

On which model would you place your bet?

Both models are significant, as said above.

PaigeMiller
Diamond | Level 26

I don't gamble, first. So I decline to answer.

 

Second, I have no idea what these models are or what they predict, I don't have your data, and you haven't provided the measures of goodness-of-fit, so I decline to answer for that reason as well.

 

 

--
Paige Miller
Ksharp
Super User

I think so.

 model #1 in some way better than #2

PaigeMiller
Diamond | Level 26

@Ksharp wrote:

I think so.

 model #1 in some way better than #2


Ok, I'll have to disagree with this, unless you are using a different meaning of "better" than I am. I want to see goodness of fit statistics, and not F-test, to determine which model fits better. Without goodness of fit measures, I can't say which model is better. If we're not talking about how well the model fits, then what definition of "better" are you using? 

--
Paige Miller
NKormanik
Barite | Level 11

Same true of t-values?

 

Say, two t-values are significant, but one is 10X larger than another.

 

Could we say that the variable with the much larger t-value is in some way more significant?

 

Does seem reasonable.

 

 

NKormanik
Barite | Level 11

What I've asked here seems like an entirely reasonable question:

 

Comparing hugely different F-values (and, t-values).

 

What can one (a very experienced expert statistician) say regarding such differences.

 

In researching, however, I came across a series of articles that seem to question the foundations, and pop the balloon:

 

https://www.tandfonline.com/toc/utas20/current

 

One example:

 

 

Good grief.

 

PaigeMiller
Diamond | Level 26

Goodness of fit measures tell you how well a model fits; not F-tests, not t-tests.

--
Paige Miller
Ksharp
Super User

Yeah. I think so.

t-value and F-value in some way are the same thing, they all measure the deviation under H0 is true .

So bigger is more significant .

But You should listen to Paige's advice . Also Check Goodness of Fitness test .

NKormanik
Barite | Level 11

Would the following be reasonable candidates for "Goodness of Fit"?  I see they are provided through Proc Mixed, which I was using.

 

Fit Statistics
 
-2 Res Log Likelihood
AIC (Smaller is Better)
AICC (Smaller is Better)
BIC (Smaller is Better)
 
Thanks!
 
Ksharp
Super User

Yes. That is right way to compare two model .

sas-innovate-white.png

Our biggest data and AI event of the year.

Don’t miss the livestream kicking off May 7. It’s free. It’s easy. And it’s the best seat in the house.

Join us virtually with our complimentary SAS Innovate Digital Pass. Watch live or on-demand in multiple languages, with translations available to help you get the most out of every session.

 

Register now!

What is Bayesian Analysis?

Learn the difference between classical and Bayesian statistical approaches and see a few PROC examples to perform Bayesian analysis in this video.

Find more tutorials on the SAS Users YouTube channel.

SAS Training: Just a Click Away

 Ready to level-up your skills? Choose your own adventure.

Browse our catalog!

Discussion stats
  • 11 replies
  • 4170 views
  • 8 likes
  • 3 in conversation