Validation of Early Warning Scores at Two Long-Term Acute Care Hospitals

Objectives: Early warning scores were developed to identify high-risk patients on the hospital wards. Research on early warning scores has focused on patients in short-term acute care hospitals, but there are other settings, such as long-term acute care hospitals, where these tools could be useful. However, the accuracy of early warning scores in long-term acute care hospitals is unknown. Design: Observational cohort study. Setting: Two long-term acute care hospitals in Illinois from January 2002 to September 2017. Patients: Admitted adult long-term acute care hospital patients. Interventions: None. Measurements and Main Results: Demographic characteristics, vital signs, laboratory values, nursing flowsheet data, and outcomes data were collected from the electronic health record. The accuracy of individual variables, the Modified Early Warning Score, the National Early Warning Score version 2, and our previously developed electronic Cardiac Arrest Risk Triage score were compared for predicting the need for acute hospital transfer or death using the area under the receiver operating characteristic curve. A total of 12,497 patient admissions were included, with 3,550 experiencing the composite outcome. The median age was 65 (interquartile range, 54-74), 46% were female, and the median length of stay in the long-term acute care hospital was 27 days (interquartile range, 17-40 d), with an 8% in-hospital mortality. Laboratory values were the best predictors, with blood urea nitrogen being the most accurate (area under the receiver operating characteristic curve, 0.63) followed by albumin, bilirubin, and WBC count (area under the receiver operating characteristic curve, 0.61). Systolic blood pressure was the most accurate vital sign (area under the receiver operating characteristic curve, 0.60). Electronic Cardiac Arrest Risk Triage (area under the receiver operating characteristic curve, 0.72) was significantly more accurate than National Early Warning Score version 2 (area under the receiver operating characteristic curve, 0.66) and Modified Early Warning Score (area under the receiver operating characteristic curve, 0.65; p < 0.01 for all pairwise comparisons). Conclusions: In this retrospective cohort study, we found that the electronic Cardiac Arrest Risk Triage score was significantly more accurate than Modified Early Warning Score and National Early Warning Score version 2 for predicting acute hospital transfer and mortality. Because laboratory values were more predictive than vital signs and the average length of stay in an long-term acute care hospital is much longer than short-term acute hospitals, developing a score specific to the long-term acute care hospital population would likely further improve accuracy, thus allowing earlier identification of high-risk patients for potentially life-saving interventions.