08-03-2014 12:31 PM
I am confused between these two terms - Sensitivity and Concordance. What i understand about these two terms -
Concordance - the number of pairs where actual 1s have higher predicted probability of becoming 1s than 0s divided by total number of pairs
Sensitivity (True Positive) - percentage of 1s correctly predicted as 1. By their definitions, both seems to be the same.
I have read somewhere Concordance = (TRUE POSITIVE + TRUE NEGATIVE) / SAMPLE SIZE. I have tried to calculate concordance using this formula. But it seems to be incorrect.
First, i have run logistic regression using PROC LOGISTIC with CTABLE PPROB=0.5 options. Then, i have calculated true positive pairs from (Sensitivity percentage* number of 1s) and true negative pairs from (Specificity percentage * number of 0s). I added them up and divided by total responses. It is not matching up with concordance score.
08-03-2014 02:28 PM
I think you have two different definitions of concordance.
If I heard "concordance" in a diagnostic setting or predictive setting, I would first think of your second definition (TRUE POSTIVE + TRUE NEGATIVE) / SAMPLE SIZE.
Sensitivity is (True Positive)/(True Positive + False Negative).
There are lost of good pages on sensitivity/specificity/PPV/NPV etc. Wikipedia has a nice 2x2 contingency table which describes different statistics.http://en.wikipedia.org/wiki/Sensitivity_and_specificity
But it looks like proc logistic is giving you something else for CONCORDANCE. It is not %agreement using sample size as the denominator. This site has a nice explanation.
Note the denominator used by this calculation is not the sample size, it's the number of pairs.
Below is sample data.
Concordance of the contingency table is (TP+TN)/SIZE = (40+50)/100= 90%.
The concordance calculated by LOGISTIC is 80.8%. The denominator for that calculation is 2475, i.e. with 45 true negatives and 55 true positive, you can have 2475 pairs.
data a; input x y weight; cards; 0 0 40 0 1 5 1 1 50 1 0 5 ; run; proc logistic data=a descending; class x; model y=x; freq weight; run; proc freq data=a ; tables x*y /measures; weight weight; run;