3rd Place Winner: CDC - 2025 Customer Recognition Awards: Innovative Problem Solver
Centers for Disease Control and Prevention
Contact: Jacques Muthusi
Country: Kenya
Award Category: Innovative Problem Solver
Tell us about the business problem you were trying to solve.
The business problem we sought to solve was the need for a standardized, efficient, and reproducible approach to evaluate diagnostic accuracy measures (DAMs) for multiple diagnostic tests across diverse settings and datasets. Diagnostic test accuracy is a critical component of evidence-based decision-making in healthcare and research. However, existing tools for evaluating diagnostic accuracy are often limited in scope, focusing on single tests or specific settings, and require significant manual effort, leading to inefficiencies and potential errors.
Our SAS macro was developed to address this gap by providing a generic, adaptable, and automated solution. The macro enables researchers and practitioners to evaluate the performance of multiple diagnostic tests simultaneously using individual-level data, regardless of the disease, diagnostic method, or study setting. It supports the generation of over 15 diagnostic accuracy measures, including sensitivity, specificity, predictive values, AUROC, AUPRC, and diagnostic odds ratios, ensuring a comprehensive evaluation of test performance.
The SAS macro was used in evaluating the accuracy and suitability of multiple diagnostic tests for monitoring HIV viral load (VL) in resource-limited settings. Traditional plasma-based VL monitoring, while highly accurate, is costly, requires specialized reagents, and must be performed in a laboratory setting by qualified technicians. This creates barriers in resource-constrained environments where such facilities and expertise are limited. Researchers wanted to assess alternative methods, specifically different types of dried blood spots (DBS) (venous, microcapillary, and direct spotting), to determine if they could serve as cost-effective, accessible, and reliable alternatives to plasma-based VL monitoring. To solve this, the SAS macro was developed to automate the analysis of multiple diagnostic tests, providing comprehensive diagnostic accuracy measures to support decision-making on adopting or rejecting these alternative methods.
Our SAS macro was developed to address this gap by providing a generic, adaptable, and automated solution. The macro enables researchers and practitioners to evaluate the performance of multiple diagnostic tests simultaneously using individual-level data, regardless of the disease, diagnostic method, or study setting. It supports the generation of over 15 diagnostic accuracy measures, including sensitivity, specificity, predictive values, AUROC, AUPRC, and diagnostic odds ratios, ensuring a comprehensive evaluation of test performance.
The SAS macro was used in evaluating the accuracy and suitability of multiple diagnostic tests for monitoring HIV viral load (VL) in resource-limited settings. Traditional plasma-based VL monitoring, while highly accurate, is costly, requires specialized reagents, and must be performed in a laboratory setting by qualified technicians. This creates barriers in resource-constrained environments where such facilities and expertise are limited. Researchers wanted to assess alternative methods, specifically different types of dried blood spots (DBS) (venous, microcapillary, and direct spotting), to determine if they could serve as cost-effective, accessible, and reliable alternatives to plasma-based VL monitoring. To solve this, the SAS macro was developed to automate the analysis of multiple diagnostic tests, providing comprehensive diagnostic accuracy measures to support decision-making on adopting or rejecting these alternative methods.
What SAS products did you use and how did you use them?
The SAS macro for evaluating multiple diagnostic tests was built using Base SAS and its core functionalities. Here's how various SAS products and features were utilized:
1. Base SAS:
Data Management: Used for importing, cleaning, and preparing individual-level diagnostic data for analysis. This included reading in datasets and transforming variables (e.g., "standard" and "test" variables).
Procedures: Utilized procedures like PROC FREQ to create 2x2 summary tables and PROC MEANS or PROC UNIVARIATE for descriptive statistics.
Macro Programming: The core automation was implemented using SAS macros, allowing the user to run analyses efficiently by specifying input parameters (datasets, thresholds, etc.).
2. SAS/STAT:
Statistical Analysis: Used to calculate advanced diagnostic accuracy measures (e.g., sensitivity, specificity, predictive values, and diagnostic odds ratio). PROC LOGISTIC procedure was leveraged for computing the area under the receiver operating characteristic (AUROC) and precision-recall curves (AUPRC).
3. SAS ODS (Output Delivery System):
Publication-Quality Output: ODS was used to export results and graphics into Word and Excel formats. This ensured high-quality, well-organized outputs suitable for research and reporting purposes.
4. SAS GRAPH (or ODS GRAPHICS):
Visualization: Overlaid AUROC and AUPRC plots were generated to visually evaluate the diagnostic test performance, assisting researchers in making informed decisions.
These SAS products worked together to streamline the process, reduce errors, and produce reproducible, publication-ready results for evaluating the diagnostic accuracy of multiple tests.
Data Management: Used for importing, cleaning, and preparing individual-level diagnostic data for analysis. This included reading in datasets and transforming variables (e.g., "standard" and "test" variables).
Procedures: Utilized procedures like PROC FREQ to create 2x2 summary tables and PROC MEANS or PROC UNIVARIATE for descriptive statistics.
Macro Programming: The core automation was implemented using SAS macros, allowing the user to run analyses efficiently by specifying input parameters (datasets, thresholds, etc.).
2. SAS/STAT:
Statistical Analysis: Used to calculate advanced diagnostic accuracy measures (e.g., sensitivity, specificity, predictive values, and diagnostic odds ratio). PROC LOGISTIC procedure was leveraged for computing the area under the receiver operating characteristic (AUROC) and precision-recall curves (AUPRC).
3. SAS ODS (Output Delivery System):
Publication-Quality Output: ODS was used to export results and graphics into Word and Excel formats. This ensured high-quality, well-organized outputs suitable for research and reporting purposes.
4. SAS GRAPH (or ODS GRAPHICS):
Visualization: Overlaid AUROC and AUPRC plots were generated to visually evaluate the diagnostic test performance, assisting researchers in making informed decisions.
These SAS products worked together to streamline the process, reduce errors, and produce reproducible, publication-ready results for evaluating the diagnostic accuracy of multiple tests.
What were the results or outcomes?
The SAS macro successfully delivered the following key outcomes:
1. Reproducibility of Published Results:
The macro was validated by reproducing results from published studies evaluating multiple types of dried blood spots (DBS) for HIV viral load (VL) monitoring compared to plasma, the gold standard. This demonstrated its accuracy and reliability for analyzing multiple diagnostic tests.
2. Comprehensive Evaluation:
The macro generated more than 15 diagnostic accuracy measures, including sensitivity, specificity, predictive values, diagnostic odds ratio (DOR), AUROC, and AUPRC for each diagnostic test. These metrics provided a detailed understanding of how well each DBS type performed relative to plasma.
3. Clear and Actionable Insights:
Publication-quality outputs, including overlaid AUROC and AUPRC graphics, were created in Word and Excel formats. These outputs made it easy for researchers to visually compare the diagnostic performance of different DBS types and make evidence-based decisions.
4. Efficiency Gains:
The macro significantly reduced analysis time by automating the creation of summary tables, statistical calculations, and graphics. This eliminated the need for manual data manipulation and reduced the risk of transcription errors.
5. Support for Resource-Limited Settings:
The analysis showed that certain types of DBS (e.g., venous or microcapillary) performed comparably to plasma for VL monitoring, highlighting their potential as cost-effective and accessible alternatives in resource-constrained environments.
6. Flexibility for Future Use:
The macro was designed to be easily customizable, allowing users to modify the source code to include additional diagnostic measures or variance estimation methods as needed.
These outcomes demonstrated that the macro is a powerful, scalable tool for diagnostic test evaluation, particularly in global health research and resource-limited settings. It supports informed decision-making and the adoption of more accessible diagnostic technologies.
The macro was validated by reproducing results from published studies evaluating multiple types of dried blood spots (DBS) for HIV viral load (VL) monitoring compared to plasma, the gold standard. This demonstrated its accuracy and reliability for analyzing multiple diagnostic tests.
2. Comprehensive Evaluation:
The macro generated more than 15 diagnostic accuracy measures, including sensitivity, specificity, predictive values, diagnostic odds ratio (DOR), AUROC, and AUPRC for each diagnostic test. These metrics provided a detailed understanding of how well each DBS type performed relative to plasma.
3. Clear and Actionable Insights:
Publication-quality outputs, including overlaid AUROC and AUPRC graphics, were created in Word and Excel formats. These outputs made it easy for researchers to visually compare the diagnostic performance of different DBS types and make evidence-based decisions.
4. Efficiency Gains:
The macro significantly reduced analysis time by automating the creation of summary tables, statistical calculations, and graphics. This eliminated the need for manual data manipulation and reduced the risk of transcription errors.
5. Support for Resource-Limited Settings:
The analysis showed that certain types of DBS (e.g., venous or microcapillary) performed comparably to plasma for VL monitoring, highlighting their potential as cost-effective and accessible alternatives in resource-constrained environments.
6. Flexibility for Future Use:
The macro was designed to be easily customizable, allowing users to modify the source code to include additional diagnostic measures or variance estimation methods as needed.
These outcomes demonstrated that the macro is a powerful, scalable tool for diagnostic test evaluation, particularly in global health research and resource-limited settings. It supports informed decision-making and the adoption of more accessible diagnostic technologies.
Why is this approach innovative?
This approach is innovative because it addresses several challenges in diagnostic test evaluation through automation, flexibility, and scalability. Here’s why it stands out:
1. Evaluation of Multiple Diagnostic Tests Simultaneously:
Unlike many existing tools that focus on a single test, this macro enables the simultaneous evaluation of multiple diagnostic tests using individual-level data. This is particularly valuable for comparative studies, saving significant time and effort.
2. Automation of Complex Analyses:
By automating the creation of 2x2 summary tables, diagnostic accuracy measures, and graphics (AUROC, AUPRC), the macro reduces manual intervention. This minimizes errors, speeds up the analysis process, and ensures consistent and reproducible results.
3. Comprehensive Diagnostic Metrics:
The macro goes beyond basic sensitivity and specificity, offering over 15 diagnostic accuracy measures, including predictive values, likelihood ratios, AUROC, AUPRC, diagnostic odds ratio, and disease prevalence. This comprehensive approach provides deeper insights for decision-making.
4. Publication-Ready Outputs:
Generating high-quality Word and Excel reports, along with overlaid AUROC and AUPRC graphics, allows researchers to present findings directly from the analysis. This streamlines the reporting process and enhances collaboration and communication of results.
5. Accessibility for Resource-Limited Settings:
The macro supports the evaluation of alternative diagnostic methods like dried blood spots (DBS), which are more affordable and accessible in low-resource environments. By enabling rigorous testing of these methods, it promotes the adoption of innovations that can improve global health equity.
6. Customizable and Reproducible:
The macro is easily modifiable, allowing users to adapt it for other diagnostic measures, data structures, or variance estimation methods. This flexibility makes it applicable to diverse research contexts and datasets.
7. Reduction of Technical Barriers:
The macro simplifies the workflow for researchers with varying levels of programming expertise. By requiring only a few input parameters (e.g., dataset, test variables, thresholds), it ensures ease of use while maintaining robust functionality.
Overall, this innovative approach bridges the gap between diagnostic test evaluation, automation, and user-friendliness, empowering researchers to conduct accurate, efficient, and reproducible analyses, especially in critical areas like global health research.
This work was published by BMC Medical Informatics and Decision Making (See Ref.). The source code for this SAS macro and data used for demonstration are available from GitHub repository at https://github.com/kmuthusi/diagnostic-testing-macro
Reference:
Muthusi, J.K., Young, P.W., Mboya, F.O. et al. %diag_test: a generic SAS macro for evaluating diagnostic accuracy measures for multiple diagnostic tests. BMC Med Inform Decis Mak 25, 21 (2025). https://doi.org/10.1186/s12911-024-02808-5
Unlike many existing tools that focus on a single test, this macro enables the simultaneous evaluation of multiple diagnostic tests using individual-level data. This is particularly valuable for comparative studies, saving significant time and effort.
2. Automation of Complex Analyses:
By automating the creation of 2x2 summary tables, diagnostic accuracy measures, and graphics (AUROC, AUPRC), the macro reduces manual intervention. This minimizes errors, speeds up the analysis process, and ensures consistent and reproducible results.
3. Comprehensive Diagnostic Metrics:
The macro goes beyond basic sensitivity and specificity, offering over 15 diagnostic accuracy measures, including predictive values, likelihood ratios, AUROC, AUPRC, diagnostic odds ratio, and disease prevalence. This comprehensive approach provides deeper insights for decision-making.
4. Publication-Ready Outputs:
Generating high-quality Word and Excel reports, along with overlaid AUROC and AUPRC graphics, allows researchers to present findings directly from the analysis. This streamlines the reporting process and enhances collaboration and communication of results.
5. Accessibility for Resource-Limited Settings:
The macro supports the evaluation of alternative diagnostic methods like dried blood spots (DBS), which are more affordable and accessible in low-resource environments. By enabling rigorous testing of these methods, it promotes the adoption of innovations that can improve global health equity.
6. Customizable and Reproducible:
The macro is easily modifiable, allowing users to adapt it for other diagnostic measures, data structures, or variance estimation methods. This flexibility makes it applicable to diverse research contexts and datasets.
7. Reduction of Technical Barriers:
The macro simplifies the workflow for researchers with varying levels of programming expertise. By requiring only a few input parameters (e.g., dataset, test variables, thresholds), it ensures ease of use while maintaining robust functionality.
Overall, this innovative approach bridges the gap between diagnostic test evaluation, automation, and user-friendliness, empowering researchers to conduct accurate, efficient, and reproducible analyses, especially in critical areas like global health research.
This work was published by BMC Medical Informatics and Decision Making (See Ref.). The source code for this SAS macro and data used for demonstration are available from GitHub repository at https://github.com/kmuthusi/diagnostic-testing-macro
Reference:
Muthusi, J.K., Young, P.W., Mboya, F.O. et al. %diag_test: a generic SAS macro for evaluating diagnostic accuracy measures for multiple diagnostic tests. BMC Med Inform Decis Mak 25, 21 (2025). https://doi.org/10.1186/s12911-024-02808-5
