Validation of the National Emergency Department Overcrowding Score (NEDOCS) in a UK non-specialist emergency department

Introduction Emergency department (ED) crowding has significant adverse consequences, however, there is no widely accepted tool to measure it. This study validated the National Emergency Department Overcrowding score (NEDOCS) (range 0-200 points), which uses routinely collected ED data. Methods This prospective single-centre study sampled data during four periods of 2018. The outcome against which NEDOCS performance was assessed was a composite of clinician opinion of crowding (physician and nurse in charge). Area under the receiver operating characteristic curves (AUROCs) and calibration plots were produced. Six-hour stratified sampling was added to adjust for temporal correlation of clinician opinion. Staff inter-rater agreement and NEDOCS association with opinion of risk, safety and staffing levels were collected. Results From 905 sampled hours, 448 paired observations were obtained, with the ED deemed crowded 18.5% of the time. Inter-rater agreement between staff was moderate (weighted kappa 0.57 (95% CI 0.56 to 0.60)). AUROC for NEDOCS was 0.81 (95% CI 0.77 to 0.86). Adjusted for temporal correlation, AUROC was 0.80 (95% CI 0.73 to 0.88). At a cut-off of 100 points sensitivity was 75.9% (95% CI 65.3% to 84.6%), specificity 72.1% (95% CI 67.1% to 76.6%), positive predictive value 38.2% (95% CI 30.7% to 46.1%) and negative predictive value 92.9% (95% CI 89.3% to 95.6%). NEDOCS underpredicted clinical opinion on Calibration assessment, only partially correcting with intercept updating. For perceived risk of harm, safety and insufficient staffing, NEDOCS AUROCs were 0.71 (95% CI 0.61 to 0.82), 0.71 (95% CI 0.63 to 0.80) and 0.70 (95% CI 0.64 to 0.76), respectively. Conclusions NEDOCS demonstrated good discriminatory power for clinical perception of crowding. Prior to implementation, determining individual unit ED cut-off point(s) would be important as published thresholds may not be generalisable. Future studies could explore refinement of existing variables or addition of new variables, including acute physiological data, which may improve performance.