topic Re: Discrepancy Between F-Test and Individual Category Significance in PROC GLM? in Statistical Procedures

Discrepancy Between F-Test and Individual Category Significance in PROC GLM?

bhr-q — Mon, 02 Sep 2024 02:22:27 GMT

Hello all,

I have an independent categorical variable named "pre_score" with 6 categories (fit, mild, moderate, very mild, managing fit, very fit) and a few confounders named (age, sex, neoro).

As shown in the attached screenshot, The overall F test in the first table is significant, and the F- test corresponding to pre_score in the second table ( source table ) is not significant, however one of the categories of pre-score in the last table is significant,

I'm unsure why this happen?

I was expected as the F-test corresponding to pre_score is not significant, there shouldn’t be any significant one for any categories of pre-score in the last table, or

if there are any significant categories like my example, I would expect the corresponding F-test to be significant as well.

proc glm data=tmp  PLOTS=DIAGNOSTICS ; 
class pre_score neuro sex;
model diff=pre_score age neuro sex/solution ss3;
run;

any input appreciated,

Re: Discrepancy Between F-Test and Individual Category Significance in PROC GLM?

Ksharp — Mon, 02 Sep 2024 03:24:34 GMT

That is a normal phenomemon .

"The overall F test in the first table is significant, "
is for the whole model . Any model would expect to significant ,unless this model is not specified right.

“and the F- test corresponding to pre_score in the second table ( source table ) is not significant, ”
is for the whole variable.

"however one of the categories of pre-score in the last table is significant,"
That is also normal. Since this variable is splited to many columns in design matrix according to its levels.
You can not expect this variable is significant once one of its level/coloum is significant.

Re: Discrepancy Between F-Test and Individual Category Significance in PROC GLM?

bhr-q — Mon, 02 Sep 2024 03:54:21 GMT

Thank you so much for your help. I tried to run the model without any confounder, and I have the attached screenshot for that; is that still a normal phenomenon?

As shown, The Overall F test is not significant, but Pairwise Comparisons show that there are a few significant differences b/n specific pairs of groups,

I was thinking as the overall F-test is not significant, there shouldn’t be any significant pairwise comparisons.

Conversely, if there are significant pairwise comparisons, one would expect the overall F-test to be significant, but as you mentioned above, my understanding is incorrect.

Thanks,

Re: Discrepancy Between F-Test and Individual Category Significance in PROC GLM?

Ksharp — Mon, 02 Sep 2024 07:37:33 GMT

Yeah. Still normal.
As I said " Overall F test " is for total data,that means the explained variance of Y by all the X variables is not significant.
a.k.a your model is not right or not specified rightly.

"very mild" variable is significant is under the assumption the model is right (" Overall F test " is significant).
this vaiable is significant does not mean " Overall F test " is significant (this vaiable's explained variance maybe be swallow or uniform by other variable).

In summary, once " Overall F test " is NOT significant, the parameter estimated is nonsense at all.

Or @StatDave_SAS maybe give you a hand.