Vision Transformer-Based Self-supervised Learning for Ulcerative Colitis Grading in Colonoscopy

Ulcerative colitis (UC) is a long-term condition that needs clinical attention and can be life-threatening. While Mayo Endoscopic Scoring is widely used to stratify patients at higher risk of developing colorectal cancer, the phenotypic endoscopic features involved in the scoring are highly inconsistent. Thus, devising automated methods is required. However, bias in the labels can also trigger such inconsistency and inaccuracy, which makes the use of fully supervised learning not preferable. We propose to exploit a self-supervised learning paradigm for automated MES grading of endoscopic images in UC. To take full advantage of local and global features, we propose to use Swin Transformers in the MoCo-v3 SSL setting. In addition, we provide a comprehensive benchmarking of other existing SSL methods. Our approach with Swin Transformer with MoCo-v3 provides performance boosts in different data size settings.