Measuring Adverse Events in Hospitalized Patients: An Administrative Method for Measuring Harm

Context: Current methods for tracking harm either require costly full manual chart review (FMCR) or rely on proxy methods that have questionable accuracy. We propose an administrative measure of harm detection that uses electronically captured data. Objective: Determine the level of agreement on harm event occurrence when harm is detected based on an administrative harm measurement tool (AHMT) compared with FMCR. Design: A retrospective chart review was used to measure the level of agreement in harm detection between an AHMT that uses electronically captured data and a FMCR. Setting: The inpatient hospital setting was used. Patients: Approximately 771 medical records from 5 hospitals were reviewed. Main Outcome Measures: Measures of positive predictive value, negative predictive value, weighted sensitivity, weighted specificity, and concordance were used to evaluate agreement between the 2 methods. Results: Although there was agreement at the harm-event level, the results were not all as high as desired: adjusted sensitivity 65%, adjusted specificity 85%, positive predictive value (PPV) 59%, negative predictive value (NPV) 88%, and concordance 75%. The patient-level results show greater agreement: adjusted sensitivity 95%, adjusted specificity 86%, PPV 61%, NPV 99%, and concordance 81%. Conclusion: The AHMT is sufficiently accurate for use as a within hospital tool to reliably detect and track harm. Nevertheless, it is not recommended as a tool to make comparisons across institutions, which has policy and payment implications. Further research using administrative harm detection, including the use of a broader set of measures and electronic health records, is needed.