BookmarkSubscribeRSS Feed
LC06
Calcite | Level 5

Hello, 

 

Looking for help and some validation on my current procedure. 

 

I have two diagnostic tests, Test1 and Test2 that I have calculated sensitivity, specificity, NPV, PPV and F1 scores for. I want to determine if one test is superior on each of these parameters. My data is given as 1 if test is positive, and 0 if test is negative. Same for disease. 

 

For sensitvity and specficity, so far I have used McNemars following this method: https://onlinecourses.science.psu.edu/stat509/node/152/

For the specficty i made sure that the first row and first column had the test negative but am getting the same P value. 

 

For PPV, and NPV I have found this macro: https://support.sas.com/resources/papers/proceedings15/2141-2015.pdf

But I can't seem to get the macro to work as it references "t" and I can't seem to figure out where the t is coming from. .

 

No clue where to start for the F1, but I've read a paper that refereces using McNemars also but not sure how that would work. 

 

Any help is greatly appreciated!

2 REPLIES 2
StatDave
SAS Super FREQ

The comparison of diagnostic tests against a gold standard can be done by comparing their ROC curves, which is composed of both sensitivity and specificity values over a range of cutpoints. This is directly available using the ROC and ROCCONTRAST statements in PROC LOGISTIC. See the example titled "Comparing Receiver Operating Characteristic Curves" in the Examples section of the LOGISTIC documentation.

LC06
Calcite | Level 5

Thanks for the response. I know ROC curves are often used for diagnostic testing comparison. But I've also seen direct comparisons of Sensitivity, specificity, NPV, PPV and F1. I've figured out the first four, but still having a hard time figuring out how to do the McNemars for F1 statistic as this paper stated they did. 

 

https://www.ncbi.nlm.nih.gov/pubmed/29425639

 

Thanks!

SAS Innovate 2025: Call for Content

Are you ready for the spotlight? We're accepting content ideas for SAS Innovate 2025 to be held May 6-9 in Orlando, FL. The call is open until September 25. Read more here about why you should contribute and what is in it for you!

Submit your idea!

What is ANOVA?

ANOVA, or Analysis Of Variance, is used to compare the averages or means of two or more populations to better understand how they differ. Watch this tutorial for more.

Find more tutorials on the SAS Users YouTube channel.

Discussion stats
  • 2 replies
  • 3559 views
  • 1 like
  • 2 in conversation