Progressive Learning With Cross-Window Consistency for Semi-Supervised Semantic Segmentation

被引:0
作者
Dang, Bo [1 ]
Li, Yansheng [1 ]
Zhang, Yongjun [1 ]
Ma, Jiayi [2 ]
机构
[1] Wuhan Univ, Sch Remote Sensing & Informat Engn, Wuhan 430079, Peoples R China
[2] Wuhan Univ, Elect Informat Sch, Wuhan 430072, Peoples R China
基金
中国国家自然科学基金;
关键词
Semantic segmentation; Semantics; Data models; Pipelines; Visualization; Remote sensing; Medical diagnostic imaging; Semi-supervised semantic segmentation; consistency loss; pseudo-label supervision;
D O I
10.1109/TIP.2024.3458854
中图分类号
TP18 [人工智能理论];
学科分类号
081104 ; 0812 ; 0835 ; 1405 ;
摘要
Semi-supervised semantic segmentation focuses on the exploration of a small amount of labeled data and a large amount of unlabeled data, which is more in line with the demands of real-world image understanding applications. However, it is still hindered by the inability to fully and effectively leverage unlabeled images. In this paper, we reveal that cross-window consistency (CWC) is helpful in comprehensively extracting auxiliary supervision from unlabeled data. Additionally, we propose a novel CWC-driven progressive learning framework to optimize the deep network by mining weak-to-strong constraints from massive unlabeled data. More specifically, this paper presents a biased cross-window consistency (BCC) loss with an importance factor, which helps the deep network explicitly constrain confidence maps from overlapping regions in different windows to maintain semantic consistency with larger contexts. In addition, we propose a dynamic pseudo-label memory bank (DPM) to provide high-consistency and high-reliability pseudo-labels to further optimize the network. Extensive experiments on three representative datasets of urban views, medical scenarios, and satellite scenes with consistent performance gain demonstrate the superiority of our framework. Our code is released at https://jack-bo1220.github.io/project/CWC.html.
引用
收藏
页码:5219 / 5231
页数:13
相关论文
共 78 条
[1]  
Berthelot D, 2019, ADV NEUR IN, V32
[2]   A Bayesian interpretation of the confusion matrix [J].
Caelen, Olivier .
ANNALS OF MATHEMATICS AND ARTIFICIAL INTELLIGENCE, 2017, 81 (3-4) :429-450
[3]   MultiSiam: Self-supervised Multi-instance Siamese Representation Learning for Autonomous Driving [J].
Chen, Kai ;
Hong, Lanqing ;
Xu, Hang ;
Li, Zhenguo ;
Yeung, Dit-Yan .
2021 IEEE/CVF INTERNATIONAL CONFERENCE ON COMPUTER VISION (ICCV 2021), 2021, :7526-7534
[4]   Encoder-Decoder with Atrous Separable Convolution for Semantic Image Segmentation [J].
Chen, Liang-Chieh ;
Zhu, Yukun ;
Papandreou, George ;
Schroff, Florian ;
Adam, Hartwig .
COMPUTER VISION - ECCV 2018, PT VII, 2018, 11211 :833-851
[5]  
Chen T, 2020, PR MACH LEARN RES, V119
[6]   Collaborative Global-Local Networks for Memory-Efficient Segmentation of Ultra-High Resolution Images [J].
Chen, Wuyang ;
Jiang, Ziyu ;
Wang, Zhangyang ;
Cui, Kexin ;
Qian, Xiaoning .
2019 IEEE/CVF CONFERENCE ON COMPUTER VISION AND PATTERN RECOGNITION (CVPR 2019), 2019, :8916-8925
[7]   Semi-Supervised Semantic Segmentation with Cross Pseudo Supervision [J].
Chen, Xiaokang ;
Yuan, Yuhui ;
Zeng, Gang ;
Wang, Jingdong .
2021 IEEE/CVF CONFERENCE ON COMPUTER VISION AND PATTERN RECOGNITION, CVPR 2021, 2021, :2613-2622
[8]   Exploring Simple Siamese Representation Learning [J].
Chen, Xinlei ;
He, Kaiming .
2021 IEEE/CVF CONFERENCE ON COMPUTER VISION AND PATTERN RECOGNITION, CVPR 2021, 2021, :15745-15753
[9]   The Cityscapes Dataset for Semantic Urban Scene Understanding [J].
Cordts, Marius ;
Omran, Mohamed ;
Ramos, Sebastian ;
Rehfeld, Timo ;
Enzweiler, Markus ;
Benenson, Rodrigo ;
Franke, Uwe ;
Roth, Stefan ;
Schiele, Bernt .
2016 IEEE CONFERENCE ON COMPUTER VISION AND PATTERN RECOGNITION (CVPR), 2016, :3213-3223
[10]   DeepGlobe 2018: A Challenge to Parse the Earth through Satellite Images [J].
Demir, Ilke ;
Koperski, Krzysztof ;
Lindenbaum, David ;
Pang, Guan ;
Huang, Jing ;
Bast, Saikat ;
Hughes, Forest ;
Tuia, Devis ;
Raskar, Ramesh .
PROCEEDINGS 2018 IEEE/CVF CONFERENCE ON COMPUTER VISION AND PATTERN RECOGNITION WORKSHOPS (CVPRW), 2018, :172-181