Clinical Performance Evaluation of a Machine Learning System for Predicting Hospital-Acquired Clostridium Difficile Infection

Clostridium difficile infection (CDI) is a common and often serious hospital-acquired infection. The CDI Risk Estimation System (CREST) was developed to apply machine learning methods to predict a patient's daily hospital-acquired CDI risk using information from the electronic health record (EHR). In recent years, several systems have been developed to predict patient health risks based on electronic medical record information. How to interpret the outputs of such systems and integrate them with healthcare work processes remains a challenge, however. In this paper, we explore the clinical interpretation of CDI Risk Scores assigned by the CREST framework for an L1-regularized Logistic Regression classifier trained using EHR data from the publicly available MIMIC-III Database. Predicted patient CDI risk is used to calculate classifier system output sensitivity, specificity, positive and negative predictive values, and diagnostic odds ratio using EHR data from five days and one day before diagnosis. We identify features which are strongly predictive of evolving infection by comparing coefficient weights for our trained models and consider system performance in the context of potential clinical applications.