autoSMIM: Automatic Superpixel-Based Masked Image Modeling for Skin Lesion Segmentation

被引:12
作者
Wang, Zhonghua [1 ,2 ]
Lyu, Junyan [1 ,3 ]
Tang, Xiaoying [1 ,2 ]
机构
[1] Southern Univ Sci & Technol, Dept Elect & Elect Engn, Shenzhen 518055, Guangdong, Peoples R China
[2] Southern Univ Sci & Technol, Jiaxing Res Inst, Jiaxing 314011, Zhejiang, Peoples R China
[3] Univ Queensland, Queensland Brain Inst, St Lucia, Qld 4072, Australia
关键词
Skin; Lesions; Image segmentation; Task analysis; Data models; Semantics; Feature extraction; Skin lesion segmentation; self-supervised learning; masked image modeling; superpixel;
D O I
10.1109/TMI.2023.3290700
中图分类号
TP39 [计算机的应用];
学科分类号
081203 ; 0835 ;
摘要
Skin lesion segmentation from dermoscopic images plays a vital role in early diagnoses and prognoses of various skin diseases. However, it is a challenging task due to the large variability of skin lesions and their blurry boundaries. Moreover, most existing skin lesion datasets are designed for disease classification, with relatively fewer segmentation labels having been provided. To address these issues, we propose a novel automatic superpixel-based masked image modeling method, named autoSMIM, in a self-supervised setting for skin lesion segmentation. It explores implicit image features from abundant unlabeled dermoscopic images. autoSMIM begins with restoring an input image with randomly masked superpixels. The policy of generating and masking superpixels is then updated via a novel proxy task through Bayesian Optimization. The optimal policy is subsequently used for training a new masked image modeling model. Finally, we finetune such a model on the downstream skin lesion segmentation task. Extensive experiments are conducted on three skin lesion segmentation datasets, including ISIC 2016, ISIC 2017, and ISIC 2018. Ablation studies demonstrate the effectiveness of superpixel-based masked image modeling and establish the adaptability of autoSMIM. Comparisons with state-of-the-art methods show the superiority of our proposed autoSMIM. The source code is available at https://github.com/Wzhjerry/autoSMIM
引用
收藏
页码:3501 / 3511
页数:11
相关论文
共 50 条
[41]   DSNet: Automatic dermoscopic skin lesion segmentation [J].
Hasan, Md Kamrul ;
Dahal, Lavsen ;
Samarakoon, Prasad N. ;
Tushar, Fakrul Islam ;
Marti, Robert .
COMPUTERS IN BIOLOGY AND MEDICINE, 2020, 120
[42]   PolSAR Image Classification With Multiscale Superpixel-Based Graph Convolutional Network [J].
Cheng, Jianda ;
Zhang, Fan ;
Xiang, Deliang ;
Yin, Qiang ;
Zhou, Yongsheng .
IEEE TRANSACTIONS ON GEOSCIENCE AND REMOTE SENSING, 2022, 60
[43]   SUPERPIXEL-BASED IMAGE INPAINTING WITH SIMPLE USER GUIDANCE [J].
Zhang, Xin ;
Hamann, Bernd ;
Pan, Xiao ;
Zhang, Caiming .
2017 24TH IEEE INTERNATIONAL CONFERENCE ON IMAGE PROCESSING (ICIP), 2017, :3785-3789
[44]   Automatic Skin Lesion Segmentation based on Saliency and Color [J].
Ramella, Giuliana .
VISAPP: PROCEEDINGS OF THE 15TH INTERNATIONAL JOINT CONFERENCE ON COMPUTER VISION, IMAGING AND COMPUTER GRAPHICS THEORY AND APPLICATIONS, VOL 4: VISAPP, 2020, :452-459
[45]   Image Segmentation with Superpixel Based Covariance Descriptor [J].
Gu, Xianbin ;
Purvis, Martin .
TRENDS AND APPLICATIONS IN KNOWLEDGE DISCOVERY AND DATA MINING (PAKDD 2016), 2016, 9794 :154-165
[46]   Superpixel Based Hierarchical Segmentation for Color Image [J].
Wu, Chong ;
Zhang, Le ;
Zhang, Houwang ;
Yan, Hong .
IEICE TRANSACTIONS ON INFORMATION AND SYSTEMS, 2020, E103D (10) :2246-2249
[47]   Brain MR Image Classification Using Superpixel-Based Deep Transfer Learning [J].
Behera, Tanmay Kumar ;
Khan, Muhammad Attique ;
Bakshi, Sambit .
IEEE JOURNAL OF BIOMEDICAL AND HEALTH INFORMATICS, 2024, 28 (03) :1218-1227
[48]   A survey on the utilization of Superpixel image for clustering based image segmentation [J].
Buddhadev Sasmal ;
Krishna Gopal Dhal .
Multimedia Tools and Applications, 2023, 82 :35493-35555
[49]   MSDUNet: A Model Based on Feature Multi-Scale and Dual-Input Dynamic Enhancement for Skin Lesion Segmentation [J].
Li, Xiaosen ;
Li, Linli ;
Xing, Xinlong ;
Liao, Huixian ;
Wang, Wenji ;
Dong, Qiutong ;
Qin, Xiao ;
Yuan, Chang'an .
IEEE TRANSACTIONS ON MEDICAL IMAGING, 2025, 44 (07) :2819-2830
[50]   Superpixel-Based Multitask Learning Framework for Hyperspectral Image Classification [J].
Jia, Sen ;
Deng, Bin ;
Zhu, Jiasong ;
Jia, Xiuping ;
Li, Qingquan .
IEEE TRANSACTIONS ON GEOSCIENCE AND REMOTE SENSING, 2017, 55 (05) :2575-2588