Why the correlation coefficient of Kendall’s Tau is small but the P value is significant?

[Dennisky]

Dear all,

We have conducted a study about an expert consulting.

And the main result is calculating the correlation coefficient is Kendall’s Tau.

There are 45 experts participated the study and scored the evaluation item (30 items).

However, the Kendall’s Tau is 0.3 and the P<0.001.

Why the correlation coefficient of Kendall’s Tau is small but the P value is significant?

Thanks a lot !

[PaigeMiller]

P value < 0.0001 indicates that the correlation of 0.3 (likely) did not happen by random chance.

[Dennisky]

Thanks a lot.

I still have a confusion about a problem that is there a standard for the correlation coefficient is Kendall’s Tau.

We know that Kendall's coefficient values can range from 0 to 1.

Kendall's coefficients of 0.9 or higher are considered very good.

But a Kendall's coefficient of 0.3 in our study is good or moderate or fair agreement range?

similar to Kappa, Is there a possible interpretation of Kendall's coefficient？

[PaigeMiller]

With respect to any correlation, the value that you get must be interpreted in terms of the data and in terms of the subject matter and in terms of the noise in the system. It also depends on whatever action you might take once you learn the value of the correlation.

In problems in physics, a correlation of 0.9 might be considered poor. In problems in social sciences, a correlation of 0.3 might be considered good.

[Rick_SAS]

It is difficult to give a general rule that applies in all situations, but Kendall's statistic tends to be related to Spearman's correlation, which is the same as the Pearson correlation on the ranks of the data.  The relationship between Kendall's tau and Spearman's rho isn't quite linear, but as a first approximation, it is roughly linear. My experience is that Kendall's tau tends to be about 0.9 (give or take) of the value of Spearman's rho.

So if you want a rule of thumb, you can use the rule that social scientists use. For Pearson's correlation, some people use

• Strong positive correlation is > 0.5
• Moderate positive correlation is > 0.3
• Weak positive correlation is > 0.1

The negative of these values are used for negative correlations:

• Strong negative correlation < -0.5
• Moderate negative correlation < -0.3
• Weak negative correlation < -0.1

You can use these values (or any others you prefer) for Spearman's correlation if you remember that you are referring to correlations of the ranks, not the data values themselves. Because of the relationship between Spearman's rho and  Kendall's tau, you can apply these values to Kendall's correlation of the ranks by multiplying by some factor less than 1, such as 0.9. Therefore, a possible set of rules for Kendall's tau is:

• Strong positive agreement between the ranks when tau > 0.45
• Moderate positive agreement between the ranks when tau > 0.27
• Weak positive agreement between the ranks when tau is > 0.09

Use negative values and less-than signs for negative correlations.

[Dennisky]

Thanks!

Your suggestion has given me a lot of inspiration, including we can use the rule that social scientists use.

However, we focus on clinical research related to medicine and nursing.

So, is there a rule that clinical meidical scientists use?

Thanks a lot

[Rick_SAS]

The primary rule is to report the statistic, the sample size, and a confidence interval. Then there is no ambiguity.

If you insist on using words such as "strong," "moderate," and "weak," there are several conventions, which can be revealed by using an internet search.

For example, Schober, Boer, and Schwarte ("Correlation Coefficients: Appropriate Use and Interpretation," Anesthesia & Analgesia, 2018) mention using the following cutpoints:

• 0 - 0.1  Negligible
• 0.1 - 0.39 Weak
• 0.4 - 0.69 Moderate
• 0.7 - 0.89 Strong
• 0.9 - 1 Very Strong

[Dennisky]

Thank you for your patient explanation and guidance. It was of great help to us.

Moreover, we are confused about this definition of the null hypothesis for Kendall W. Please refer to the response for more details as follow.

[PaigeMiller]

---

@Dennisky wrote:

Your suggestion has given me a lot of inspiration, including we can use the rule that social scientists use.

However, we focus on clinical research related to medicine and nursing.

So, is there a rule that clinical meidical scientists use?

---

You should look at published papers in your field (clinical research related to medicine and nursing) and see what kinds of correlations are reported in those papers. If, for example, the papers show correlations of 0.5 and 0.6, then a 0.3 is not good. On the other hand, if those papers show correlations of 0.3, then yours fits right in.

[Dennisky]

Thanks a lot !

We have calculated the kendall w value by SAS, R and SPSS, respectively.

The results calculated by the three software are  the same.

Afterthat, we assume that everyone(raters) has given exactly the same scores to all of the items.

The result of SAS is also consisitent with R.( the result of kendall w value and p value is NA)

But we were surprised to find that the results of SPSS software were very different from those of SAS and R.

(see supplment figure,the three pictures in order are the results of SPSS, SAS, and R.)

So,the definition of null hypothesis is confusing me.
Did SPSS defines null hypothesis for Kendall W as “The distributions of variables are the same” ?
If it is true, this definition look completely opposite to the way our defined the null hypothesis “there is no agreement among raters”.

How to understand this result?

 

Thanks!

[Rick_SAS]

Kendall was a prolific researcher who contributed not one but TWO statistics that bears his name:

1. The Kendall rank correlation (called 'tau') is a measure of association. The null hypothesis for the test statistic is whether two variables are independent versus the alternative, which is that the variables are associated.  This is the quantity that we have been talking about.
2. The Kendall concordance (called 'W') is used for inter-rater reliability. The null hypothesis is whether the raters are ranking randomly versus the alternative that the rankings bewteen raters are associated.

[Dennisky]

We made a mistake in our writing.

It has always been Kendall's W here, not Kendall’s Tau.

I am very sorry and  I appreciate you bringing up this mistake.

That is, the analysis we have been conducting throughout is the Kendall's W analysis by three software.

Thus, we are confusing with the result conducted by SPSS.

[Rick_SAS · Accepted solution]

I see. So the title "Why the correlation coefficient of Kendall’s Tau..." is also misleading, and all of our previous responses are irrelevant.

For two raters, you can use the AGREE option on the TABLES statement on PROC FREQ. For multiple raters, see the %MAGREE macro.

Good luck with your problem.

[Dennisky]

Thank you very much. 
The methods provided on this website (the %MAGREE macro.)have been of great assistance to us.

[PaigeMiller]

---

@Dennisky wrote:

We have calculated the kendall w value by SAS, R and SPSS, respectively.

The results calculated by the three software are  the same.

---

This is not what I suggested at all. And it adds no value in determining whether or not 0.3 is a "good" correlation or a "poor" correlation or somewhere in the middle.