Endoscopy-Driven Pretraining for Classification of Dysplasia in Barrett's Esophagus with Endoscopic Narrow-Band Imaging Zoom Videos

Featured Application This work focuses on the use of deep learning and convolutional neural networks to classify dysplasia in patients with Barrett's Esophagus. For this, we use the Narrow Band Imaging modality, which exploits different wavelengths to capture the tissue at different levels of penetration depth, leading to high-contrast imaging of mucosal and vascular patterns. Such patterns reveal information on histology, but they are hard to interpret for physicians. Our approach will aid the endoscopist in the interpretation of NBI imagery, leading to a higher false positive detections and a more robust diagnosis. In addition, this work also shows the potential benefits of using endoscopy-driven pretraining, instead of the more commonly used natural-image pretraining based on e.g., ImageNet. Abstract Endoscopic diagnosis of early neoplasia in Barrett's Esophagus is generally a two-step process of primary detection in overview, followed by detailed inspection of any visible abnormalities using Narrow Band Imaging (NBI). However, endoscopists struggle with evaluating NBI-zoom imagery of subtle abnormalities. In this work, we propose the first results of a deep learning system for the characterization of NBI-zoom imagery of Barrett's Esophagus with an accuracy, sensitivity, and specificity of 83.6%, 83.1%, and 84.0%, respectively. We also show that endoscopy-driven pretraining outperforms two models, one without pretraining as well as a model with ImageNet initialization. The final model outperforms absence of pretraining by approximately 10% and the performance is 2% higher in terms of accuracy compared to ImageNet pretraining. Furthermore, the practical deployment of our model is not hampered by ImageNet licensing, thereby paving the way for clinical application.