Quantifying the Accuracy of Clinician Risk Assessment for Postpartum Hemorrhage

ObjectiveTo measure the accuracy of postpartum hemorrhage (PPH) risk assessment performed by unaided individual clinicians, to inform future comparison to alternative risk assessment methods.MethodsProspective PPH risk assessments were collected from obstetric care team clinicians at two quaternary medical centers in the United States (Vanderbilt University Medical Center, Brigham and Women's Hospital) from January 2022 to January 2023, following written informed consent from the providers. The data included a cohort of both vaginal and cesarean deliveries (CD). For each assessment, the clinician quantified the patient's predicted PPH risk on a scale from 0 to 100% and rated their confidence in these assessments using a 5-point Likert scale, ranging from 'not at all confident' to 'completely confident'. Medical records were reviewed 24 hours postpartum to assess the dichotomous outcome of PPH, defined as blood loss >= 1000 mL. The accuracy of these predictions was evaluated using the area under the receiver operating characteristic curve (AUC).ResultsOf 271 patients, 32 (11.8%) experienced PPH, accounting for 11.4% (104/915) of assessments. The overall AUC was 0.64 (95% confidence interval (CI): 0.58-0.71). Prediction accuracy was higher for CD than for vaginal deliveries, with AUCs of 0.82 (95% CI: 0.72-0.91) and 0.56 (95% CI: 0.48-0.63), respectively. No significant differences in the accuracy of assessments were observed according to physician specialty, physician experience level, or confidence level of the assessment.ConclusionOverall unaided clinician performance in predicting PPH was moderate, with an AUC of 0.64. Predictions were more accurate for patients undergoing CD. Further study is needed to understand how clinician performance compares to other modalities of risk prediction.