Advanced Receiver Operating Characteristic Curve Analysis to Identify Outliers in Binary Machine Learning Classifications for Precision Medicine

The Receiver Operating Characteristic (ROC) curve is a critical tool for binary classification analysis in medicine, with the Area Under the ROC Curve (AUROC) serving as a widely accepted metric to evaluate the performance of binary classifiers. This study conducts a comprehensive review of the ROC curve with a focus on its utility in outlier identification. We introduce a novel scoring method to rank actual positives and actual negatives within a test set, according to their impact on AUROC degradation. We bridge the scoring system with the ROC curve analysis to quantify each data point's contribution to AUROC loss. Furthermore, we introduce the IMICS ROC Analyzer, a graphical user interface-based software, embedded with our innovative algorithms. Through the use of an open-source prostate cancer dataset, we illustrate the application of our algorithms for practical outlier detection in binary classification tasks. The IMICS ROC Analyzer enhances the field of precision medicine by allowing for measuring an individual's contributions (be it patients, lesions, or samples) to the overall AUROC, thus facilitating confidence measurement of Machine Learning (ML) classifiers for individual cases of interest in a cohort.