BookmarkSubscribeRSS Feed
LC06
Calcite | Level 5

Hello, 

 

Looking for help and some validation on my current procedure. 

 

I have two diagnostic tests, Test1 and Test2 that I have calculated sensitivity, specificity, NPV, PPV and F1 scores for. I want to determine if one test is superior on each of these parameters. My data is given as 1 if test is positive, and 0 if test is negative. Same for disease. 

 

For sensitvity and specficity, so far I have used McNemars following this method: https://onlinecourses.science.psu.edu/stat509/node/152/

For the specficty i made sure that the first row and first column had the test negative but am getting the same P value. 

 

For PPV, and NPV I have found this macro: https://support.sas.com/resources/papers/proceedings15/2141-2015.pdf

But I can't seem to get the macro to work as it references "t" and I can't seem to figure out where the t is coming from. .

 

No clue where to start for the F1, but I've read a paper that refereces using McNemars also but not sure how that would work. 

 

Any help is greatly appreciated!

2 REPLIES 2
StatDave
SAS Super FREQ

The comparison of diagnostic tests against a gold standard can be done by comparing their ROC curves, which is composed of both sensitivity and specificity values over a range of cutpoints. This is directly available using the ROC and ROCCONTRAST statements in PROC LOGISTIC. See the example titled "Comparing Receiver Operating Characteristic Curves" in the Examples section of the LOGISTIC documentation.

LC06
Calcite | Level 5

Thanks for the response. I know ROC curves are often used for diagnostic testing comparison. But I've also seen direct comparisons of Sensitivity, specificity, NPV, PPV and F1. I've figured out the first four, but still having a hard time figuring out how to do the McNemars for F1 statistic as this paper stated they did. 

 

https://www.ncbi.nlm.nih.gov/pubmed/29425639

 

Thanks!

sas-innovate-2024.png

Available on demand!

Missed SAS Innovate Las Vegas? Watch all the action for free! View the keynotes, general sessions and 22 breakouts on demand.

 

Register now!

What is ANOVA?

ANOVA, or Analysis Of Variance, is used to compare the averages or means of two or more populations to better understand how they differ. Watch this tutorial for more.

Find more tutorials on the SAS Users YouTube channel.

Discussion stats
  • 2 replies
  • 3352 views
  • 1 like
  • 2 in conversation