topic Re: tdiff - Is this just a Student's t-test? in Statistical Procedures

tdiff - Is this just a Student's t-test?

pricek — Mon, 01 Dec 2014 20:47:01 GMT

Hi All,

I am working with PROC MIXED and I am just getting confused on what certain syntax means/what tests are being completed.

I am trying to complete a PROC MIXED model, comparing lsmeans. For the lsmeans my coding is: lsmeans vx*flr*region*chall / cl tdiff; (where the vx*flr*region*chall is my interaction term that I am looking at to do comparisons).

Is the tdiff coding doing a Student's t-test for pair-wise comparisons? I have a lot of p-values and this seems like what it is (and if it is the default), I just want to confirm.

Additionally, how does this compare to using "adjust=Tukey" especially with interaction terms? I find that the p-values are very different. My alpha value for the tdiff is 0.05 and it is the same for the Tukey test.

Thank you for your help!

Re: tdiff - Is this just a Student's t-test?

PaigeMiller — Mon, 01 Dec 2014 21:40:25 GMT

In PROC MIXED, the tdiff might be considered to be a relative of the Student's t-test, but honestly, its a different test in most situations (although there could be special cases where it is exactly equal to the t-test). With a 4-way interaction, you get all possible pairs, but the error terms used probably would not be the same as the error term if you were simply using a Student's t-test on each pair.

Re: tdiff - Is this just a Student's t-test?

lvm — Tue, 02 Dec 2014 15:56:01 GMT

You are getting t statistics with that option. That is, you are seeing the pairwise difference of estimated expected values divided by the estimated standard error of the difference. This has a Student t distribution under the null hypothesis of no difference. I think Paige's point is that if you pulled out just the observations for one set of factor levels (throwing out the rest), and did a traditional t test, you would not get the same t statistic (this is true). Nevertheless, you are looking at t statistics with the MIXED output. It is debatable which adjustment for multiplicity that one should use. There is now a lot of support to use the adjust=simulate option, because this is based on fewer assumptions.

Re: tdiff - Is this just a Student's t-test?

pricek — Tue, 02 Dec 2014 16:14:39 GMT

Thank you all for your help. So then when using an interaction term would tdiff be better than Tukey, or what needs to be considered with the data? In my situation I run into the problem of significant p-values (<0.05) with the tdiff test versus non-significant p-values (<0.05) with the Tukey test...where the number biologically make sense to be significant. What makes the tdiff test so different from adjust=tukey that the p-values can go from significant to non-significant, respectively.

Also, can the tdiff test officially be called a t-statistic? Or a multiple comparison t-statistic?

Thank you all again for some insightful answers.

Re: tdiff - Is this just a Student's t-test?

PaigeMiller — Tue, 02 Dec 2014 16:50:49 GMT

tdiff be better than Tukey, or what needs to be considered with the data?

These are different tests, one is not better than the other unless you specify exactly what the intended use of the test is.

In my situation I run into the problem of significant p-values (<0.05) with the tdiff test versus non-significant p-values (<0.05) with the Tukey test...where the number biologically make sense to be significant.

Tukey and tdiff should be different. I can't say which one is better for your purposes, because I don't really know what your trying to show. And I don't understand "where the number biologically make sense to be significant", statistical testing is empirical and we do not know the answer beforehand. If you know the answer beforehand, then you don't want to do empirical statistical testing.

Also, can the tdiff test officially be called a t-statistic? Or a multiple comparison t-statistic?

I don't have a problem calling it a t-statistic. It is not a multiple comparison t-statistic unless you do other things to it to make it a multiple comparison t-statistic.