topic Re: Inconsistency Between Type 3 Test and Pairwise Comparisons in PROC MIXED in Statistical Procedures

Inconsistency Between Type 3 Test and Pairwise Comparisons in PROC MIXED

Yui_1122 — Thu, 19 Sep 2024 16:31:09 GMT

Hello,

I'm encountering inconsistencies in results between Type 3 Tests of Fixed Effects and pairwise comparisons when using proc mixed.

My dataset consists of 5 subjects, each measured at 3 different time points. There are 15 observations in total.

I'm fitting a linear mixed-effects model to examine mean value differences among three groups. The Type 3 Test indicates a significant difference among at least one pair of groups (P-value = 0.0028), but none of the pairwise comparisons show significance (their p-values are 0.2096, 0.7895, and 0.8126). This discrepancy is puzzling to me.

For the variance-covariance structure, I've chosen the UN over CS and TOEP based on assumptions and AIC, but this inconsistency is specific to UN only and not observed with CS or TOEP.

Below is my SAS code. Any insights into this issue would be greatly appreciated!

proc mixed data=data plots=residualpanel;
class id group;
model outcome = group / s cl residual;
repeated group / subject=id type=UN r rcorr;
lsmeans group / cl diff;
run;

Thank you!!

Re: Inconsistency Between Type 3 Test and Pairwise Comparisons in PROC MIXED

PaigeMiller — Thu, 19 Sep 2024 16:41:25 GMT

This was discussed a few days ago here in the SAS Communities.

https://communities.sas.com/t5/SAS-Procedures/Proc-Glimmix-Type-III-test-and-LSMeans-do-not-agree/td-p/147046

Re: Inconsistency Between Type 3 Test and Pairwise Comparisons in PROC MIXED

Yui_1122 — Thu, 19 Sep 2024 18:28:14 GMT

Hi Paige,

Thank you for your prompt reply!

However, my question differs from the one in the link you shared. In that case, the person observed both a significant Type 3 effect and confidence interval for one of the pairwise comparisons, which seems consistent. In my situation, I have only 3 groups, and I used LSMEANS for pairwise comparisons along with the Type 3 p-value. While the overall test result is significant, none of the 3 pairwise comparisons are. Additionally, my sample size is too small (n=15) to adjust for any interaction term. Moreover, my question is more focused on the discrepancies that arise from selecting different variance-covariance structures.

Re: Inconsistency Between Type 3 Test and Pairwise Comparisons in PROC MIXED

PaigeMiller — Thu, 19 Sep 2024 18:57:30 GMT

Overall tests use different logic and different math and answer different questions than the pairwise test. There is no reason to expect that the results will be consistent 100% of the time. Some of the time they will be consistent, other times they will not be consistent.

Re: Inconsistency Between Type 3 Test and Pairwise Comparisons in PROC MIXED

Ksharp — Fri, 20 Sep 2024 01:21:22 GMT

That is two total different questions.
GROUP is significant means this variable can explain more/enough variance of Y variable in the model.

“lsmeans group” is checking the expected value/mean of different group value is different or not.
That is totally two differnt concept with the above.