Deep Learning in Diagnosis of Maxillary Sinusitis Using Conventional Radiography

Objectives The aim of this study was to compare the diagnostic performance of a deep learning algorithm with that of radiologists in diagnosing maxillary sinusitis on Waters' view radiographs. Materials and Methods Among 80,475 Waters' view radiographs, examined between May 2003 and February 2017, 9000 randomly selected cases were classified as normal or maxillary sinusitis based on radiographic findings and divided into training (n = 8000) and validation (n = 1000) sets to develop a deep learning algorithm. Two test sets composed of Waters' view radiographs with concurrent paranasal sinus computed tomography were labeled based on computed tomography findings: one with temporal separation (n = 140) and the other with geographic separation (n = 200) from the training set. Area under the receiver operating characteristics curve (AUC), sensitivity, and specificity of the algorithm and 5 radiologists were assessed. Interobserver agreement between the algorithm and majority decision of the radiologists was measured. The correlation coefficient between the predicted probability of the algorithm and average confidence level of the radiologists was determined. Results The AUCs of the deep learning algorithm were 0.93 and 0.88 for the temporal and geographic external test sets, respectively. The AUCs of the radiologists were 0.83 to 0.89 for the temporal and 0.75 to 0.84 for the geographic external test sets. The deep learning algorithm showed statistically significantly higher AUC than radiologist in both test sets. In terms of sensitivity and specificity, the deep learning algorithm was comparable to the radiologists. A strong interobserver agreement was noted between the algorithm and radiologists (Cohen kappa coefficient, 0.82). The correlation coefficient between the predicted probability of the algorithm and confidence level of radiologists was 0.89 and 0.84 for the 2 test sets, respectively. Conclusions The deep learning algorithm could diagnose maxillary sinusitis on Waters' view radiograph with superior AUC and comparable sensitivity and specificity to those of radiologists.