topic Why the correlation coefficient of Kendall’s Tau is small but the P value is significant? in Statistical Procedures

Why the correlation coefficient of Kendall’s Tau is small but the P value is significant?

Dennisky — Thu, 16 Mar 2023 07:31:13 GMT

Dear all,

We have conducted a study about an expert consulting.

And the main result is calculating the correlation coefficient is Kendall’s Tau.

There are 45 experts participated the study and scored the evaluation item (30 items).

However, the Kendall’s Tau is 0.3 and the P<0.001.

Why the correlation coefficient of Kendall’s Tau is small but the P value is significant?

Thanks a lot !

Re: Why the correlation coefficient of Kendall’s Tau is small but the P value is significant?

PaigeMiller — Thu, 16 Mar 2023 10:19:28 GMT

P value < 0.0001 indicates that the correlation of 0.3 (likely) did not happen by random chance.

Re: Why the correlation coefficient of Kendall’s Tau is small but the P value is significant?

Dennisky — Fri, 17 Mar 2023 02:00:33 GMT

Thanks a lot.

I still have a confusion about a problem that is there a standard for the correlation coefficient is Kendall’s Tau.

We know that Kendall's coefficient values can range from 0 to 1.

Kendall's coefficients of 0.9 or higher are considered very good.

But a Kendall's coefficient of 0.3 in our study is good or moderate or fair agreement range?

similar to Kappa, Is there a possible interpretation of Kendall's coefficient？

Re: Why the correlation coefficient of Kendall’s Tau is small but the P value is significant?

PaigeMiller — Fri, 17 Mar 2023 10:29:57 GMT

With respect to any correlation, the value that you get must be interpreted in terms of the data and in terms of the subject matter and in terms of the noise in the system. It also depends on whatever action you might take once you learn the value of the correlation.

In problems in physics, a correlation of 0.9 might be considered poor. In problems in social sciences, a correlation of 0.3 might be considered good.

Re: Why the correlation coefficient of Kendall’s Tau is small but the P value is significant?

Rick_SAS — Sat, 18 Mar 2023 10:46:38 GMT

It is difficult to give a general rule that applies in all situations, but Kendall's statistic tends to be related to Spearman's correlation, which is the same as the Pearson correlation on the ranks of the data. The relationship between Kendall's tau and Spearman's rho isn't quite linear, but as a first approximation, it is roughly linear. My experience is that Kendall's tau tends to be about 0.9 (give or take) of the value of Spearman's rho.

So if you want a rule of thumb, you can use the rule that social scientists use. For Pearson's correlation, some people use

Strong positive correlation is > 0.5
Moderate positive correlation is > 0.3
Weak positive correlation is > 0.1

The negative of these values are used for negative correlations:

Strong negative correlation < -0.5
Moderate negative correlation < -0.3
Weak negative correlation < -0.1

You can use these values (or any others you prefer) for Spearman's correlation if you remember that you are referring to correlations of the ranks, not the data values themselves. Because of the relationship between Spearman's rho and Kendall's tau, you can apply these values to Kendall's correlation of the ranks by multiplying by some factor less than 1, such as 0.9. Therefore, a possible set of rules for Kendall's tau is:

Strong positive agreement between the ranks when tau > 0.45
Moderate positive agreement between the ranks when tau > 0.27
Weak positive agreement between the ranks when tau is > 0.09

Use negative values and less-than signs for negative correlations.

Re: Why the correlation coefficient of Kendall’s Tau is small but the P value is significant?

Dennisky — Sun, 19 Mar 2023 08:16:53 GMT

Thanks!

Your suggestion has given me a lot of inspiration, including we can use the rule that social scientists use.

However, we focus on clinical research related to medicine and nursing.

So, is there a rule that clinical meidical scientists use?

Thanks a lot

Re: Why the correlation coefficient of Kendall’s Tau is small but the P value is significant?

Rick_SAS — Sun, 19 Mar 2023 10:48:11 GMT

The primary rule is to report the statistic, the sample size, and a confidence interval. Then there is no ambiguity.

If you insist on using words such as "strong," "moderate," and "weak," there are several conventions, which can be revealed by using an internet search.

For example, Schober, Boer, and Schwarte ("Correlation Coefficients: Appropriate Use and Interpretation," Anesthesia & Analgesia, 2018) mention using the following cutpoints:

0 - 0.1 Negligible
0.1 - 0.39 Weak
0.4 - 0.69 Moderate
0.7 - 0.89 Strong
0.9 - 1 Very Strong

Re: Why the correlation coefficient of Kendall’s Tau is small but the P value is significant?

PaigeMiller — Sun, 19 Mar 2023 13:22:09 GMT

@Dennisky wrote:

Your suggestion has given me a lot of inspiration, including we can use the rule that social scientists use.

However, we focus on clinical research related to medicine and nursing.

So, is there a rule that clinical meidical scientists use?

You should look at published papers in your field (clinical research related to medicine and nursing) and see what kinds of correlations are reported in those papers. If, for example, the papers show correlations of 0.5 and 0.6, then a 0.3 is not good. On the other hand, if those papers show correlations of 0.3, then yours fits right in.

Re: Why the correlation coefficient of Kendall’s Tau is small but the P value is significant?

Dennisky — Tue, 21 Mar 2023 06:49:54 GMT

Thanks a lot !

We have calculated the kendall w value by SAS, R and SPSS, respectively.

The results calculated by the three software are the same.

Afterthat, we assume that everyone(raters) has given exactly the same scores to all of the items.

The result of SAS is also consisitent with R.( the result of kendall w value and p value is NA)

But we were surprised to find that the results of SPSS software were very different from those of SAS and R.

(see supplment figure,the three pictures in order are the results of SPSS, SAS, and R.)

So,the definition of null hypothesis is confusing me.
Did SPSS defines null hypothesis for Kendall W as “The distributions of variables are the same” ?
If it is true, this definition look completely opposite to the way our defined the null hypothesis “there is no agreement among raters”.

How to understand this result?

Thanks!

Re: Why the correlation coefficient of Kendall’s Tau is small but the P value is significant?

Dennisky — Tue, 21 Mar 2023 07:45:07 GMT

Thank you for your patient explanation and guidance. It was of great help to us.

Moreover, we are confused about this definition of the null hypothesis for Kendall W. Please refer to the response for more details as follow.

Re: Why the correlation coefficient of Kendall’s Tau is small but the P value is significant?

Rick_SAS — Tue, 21 Mar 2023 09:42:26 GMT

Kendall was a prolific researcher who contributed not one but TWO statistics that bears his name:

The Kendall rank correlation (called 'tau') is a measure of association. The null hypothesis for the test statistic is whether two variables are independent versus the alternative, which is that the variables are associated. This is the quantity that we have been talking about.
The Kendall concordance (called 'W') is used for inter-rater reliability. The null hypothesis is whether the raters are ranking randomly versus the alternative that the rankings bewteen raters are associated.

Re: Why the correlation coefficient of Kendall’s Tau is small but the P value is significant?

PaigeMiller — Tue, 21 Mar 2023 10:00:29 GMT

@Dennisky wrote:

We have calculated the kendall w value by SAS, R and SPSS, respectively.

The results calculated by the three software are the same.

This is not what I suggested at all. And it adds no value in determining whether or not 0.3 is a "good" correlation or a "poor" correlation or somewhere in the middle.

Re: Why the correlation coefficient of Kendall’s Tau is small but the P value is significant?

Dennisky — Tue, 21 Mar 2023 13:19:06 GMT

We made a mistake in our writing.

It has always been Kendall's W here, not Kendall’s Tau.

I am very sorry and I appreciate you bringing up this mistake.

That is, the analysis we have been conducting throughout is the Kendall's W analysis by three software.

Thus, we are confusing with the result conducted by SPSS.

Re: Why the correlation coefficient of Kendall’s Tau is small but the P value is significant?

Rick_SAS — Tue, 21 Mar 2023 13:21:09 GMT

I see. So the title "Why the correlation coefficient of Kendall’s Tau..." is also misleading, and all of our previous responses are irrelevant.

For two raters, you can use the AGREE option on the TABLES statement on PROC FREQ. For multiple raters, see the %MAGREE macro.

Good luck with your problem.

Re: Why the correlation coefficient of Kendall’s Tau is small but the P value is significant?

Dennisky — Tue, 21 Mar 2023 13:26:06 GMT

Thanks, We completely agree with your viewpoint. Coincidentally, the published values in the literature happen to be around 0.3.

On the other hand, we conducted the kendall w analysis by three software.

And as we methioned abov, we are confusing with the result from spss.

Re: Why the correlation coefficient of Kendall’s Tau is small but the P value is significant?

Dennisky — Wed, 22 Mar 2023 06:34:09 GMT

Thank you very much.
The methods provided on this website (the %MAGREE macro.)have been of great assistance to us.

Re: Why the correlation coefficient of Kendall’s Tau is small but the P value is significant?

NBrown1 — Sat, 28 Oct 2023 13:08:10 GMT

Hello,

I wonder if you had a reference for the possible set of rules outlined here for Kenduall's tau?

Kind regards,

Naomi

Re: Why the correlation coefficient of Kendall’s Tau is small but the P value is significant?

Rick_SAS — Sun, 29 Oct 2023 09:47:56 GMT

Yes.

Wicklin, R. (2023) "Weak or strong? How to interpret a Spearman or Kendall correlation", The DO Loop blog, URL https://blogs.sas.com/content/iml/2023/04/05/interpret-spearman-kendall-corr.html, accessed 29OCT2023.

The blog post has additional discussion and references,