Self-supervised learning and semi-supervised learning for multi-sequence medical image classification

Multi-sequence medical images are playing an increasingly important role in disease diagnosis because different sequences can provide complementary information. At the same time, deep learning-based methods have been widely used in computer aided diagnosis, but most of the recent success profoundly relies on large amounts of carefully labeled data, which is time-consuming and costly, especially when multiple sequences need to be labeled. To reduce the human effort of labeling multi-sequence medical images, we present a new self-supervised learning method MI-SelfL, a new semi-supervised learning method MI-SemiL, and a combined method MI-SESEL, and all these methods can exploit unlabeled data by exploring the intrinsic relation and the complementarity between multi-sequence images. We con-ducted extensive experiments on two tasks, hepatocellular carcinoma grading using dynamic contrast enhanced Magnetic Resonance Imaging (MRI) and prostate cancer classification using multiparametric MRI. The results show that compared with the fully-supervised learning baseline, MI-SelfL and MI-SemiL can both improve the model performance, whereas the combined method MI-SESEL can further improve it. (c) 2022 Elsevier B.V. All rights reserved.