topic Re: How to compare Sensitivity, specificity, PPV, NPV between two diagnostic tests in Statistical Procedures

How to compare Sensitivity, specificity, PPV, NPV between two diagnostic tests

LC06 — Thu, 13 Sep 2018 00:51:16 GMT

Hello,

Looking for help and some validation on my current procedure.

I have two diagnostic tests, Test1 and Test2 that I have calculated sensitivity, specificity, NPV, PPV and F1 scores for. I want to determine if one test is superior on each of these parameters. My data is given as 1 if test is positive, and 0 if test is negative. Same for disease.

For sensitvity and specficity, so far I have used McNemars following this method: https://onlinecourses.science.psu.edu/stat509/node/152/

For the specficty i made sure that the first row and first column had the test negative but am getting the same P value.

For PPV, and NPV I have found this macro: https://support.sas.com/resources/papers/proceedings15/2141-2015.pdf

But I can't seem to get the macro to work as it references "t" and I can't seem to figure out where the t is coming from. .

No clue where to start for the F1, but I've read a paper that refereces using McNemars also but not sure how that would work.

Any help is greatly appreciated!

Re: How to compare Sensitivity, specificity, PPV, NPV between two diagnostic tests

StatDave — Thu, 13 Sep 2018 20:23:38 GMT

The comparison of diagnostic tests against a gold standard can be done by comparing their ROC curves, which is composed of both sensitivity and specificity values over a range of cutpoints. This is directly available using the ROC and ROCCONTRAST statements in PROC LOGISTIC. See the example titled "Comparing Receiver Operating Characteristic Curves" in the Examples section of the LOGISTIC documentation.

Re: How to compare Sensitivity, specificity, PPV, NPV between two diagnostic tests

LC06 — Fri, 14 Sep 2018 01:07:16 GMT

Thanks for the response. I know ROC curves are often used for diagnostic testing comparison. But I've also seen direct comparisons of Sensitivity, specificity, NPV, PPV and F1. I've figured out the first four, but still having a hard time figuring out how to do the McNemars for F1 statistic as this paper stated they did.

https://www.ncbi.nlm.nih.gov/pubmed/29425639

Thanks!