Predicting Pouch of Douglas Obliteration Using Ultrasound and Laparoscopic Video Sets: An Interobserver and Diagnostic Accuracy Study

Objectives To investigate the diagnostic accuracy and interobserver agreement among sonologists when assessing offline ultrasound (US) video sets of the "sliding sign" and among gynecologic surgeons when assessing corresponding laparoscopic video sets to predict pouch of Douglas (POD) obliteration and to compare the performance of the groups. Methods A diagnostic and reproducibility study was conducted, including 15 observers in 4 groups: (1) senior sonologists, (2) junior sonologists, (3) general gynecologists, and (4) advanced laparoscopists. The sonologists viewed 25 offline preoperative US video sets of the sliding sign, and the surgeons viewed the corresponding intraoperative laparoscopic videos of the same patients. Each observer was asked to classify POD obliteration in the video sets and was compared to the reference standard POD state determined at real-time laparoscopy by a single investigator (G.C.). The interobserver correlation and diagnostic accuracy were evaluated among the 15 observers and 4 groups. The Cohen kappa coefficient and Fleiss kappa coefficient were used for the analysis. Results The overall accuracy, sensitivity, specificity, positive predictive value, and negative predictive value for senior sonologists were 93.3%, 100%, 89.6%, 84.4%, and 100%, respectively; for junior sonologists, 70.0%, 88.9%, 59.4%, 55.2%, and 90.5%; for general gynecologists, 75.2%, 88.1%, 78.1%, 69.8%, and 91.9%; and for advanced laparoscopists, 82.4%, 91.9%, 90.8%, 82.9%, and 95.8%. The overall agreement between senior sonologists was almost perfect (Fleiss kappa = 0.876); for junior sonologists and general gynecologists, it was moderate (Fleiss kappa = 0.589 and 0.528); and for advanced laparoscopists, it was substantial (Fleiss kappa = 0.652). Conclusions Interobserver agreement was superior among senior sonologists. Prediction of POD obliteration using offline US videos by senior sonologists is comparable to offline assessments of laparoscopic videos by advanced laparoscopists for prediction of POD obliteration.