Co-Training for Unsupervised Domain Adaptation of Semantic Segmentation Models

被引:6
作者
Gomez, Jose L. [1 ,2 ]
Villalonga, Gabriel [1 ]
Lopez, Antonio M. [1 ,2 ]
机构
[1] Univ Autonoma Barcelona UAB, Comp Vis Ctr CVC, Bellaterra 08193, Spain
[2] Univ Autonoma Barcelona UAB, Comp Sci Dept, Bellaterra 08193, Spain
关键词
domain adaptation; semi-supervised learning; semantic segmentation; autonomous driving;
D O I
10.3390/s23020621
中图分类号
O65 [分析化学];
学科分类号
070302 ; 081704 ;
摘要
Semantic image segmentation is a core task for autonomous driving, which is performed by deep models. Since training these models draws to a curse of human-based image labeling, the use of synthetic images with automatically generated labels together with unlabeled real-world images is a promising alternative. This implies addressing an unsupervised domain adaptation (UDA) problem. In this paper, we propose a new co-training procedure for synth-to-real UDA of semantic segmentation models. It performs iterations where the (unlabeled) real-world training images are labeled by intermediate deep models trained with both the (labeled) synthetic images and the real-world ones labeled in previous iterations. More specifically, a self-training stage provides two domain-adapted models and a model collaboration loop allows the mutual improvement of these two models. The final semantic segmentation labels (pseudo-labels) for the real-world images are provided by these two models. The overall procedure treats the deep models as black boxes and drives their collaboration at the level of pseudo-labeled target images, i.e., neither modifying loss functions is required, nor explicit feature alignment. We test our proposal on standard synthetic and real-world datasets for onboard semantic segmentation. Our procedure shows improvements ranging from approximately 13 to 31 mIoU points over baselines.
引用
收藏
页数:28
相关论文
共 50 条
[41]   Unsupervised Domain Adaptation Semantic Segmentation for Remote-Sensing Images via Covariance Attention [J].
Liu, Yikun ;
Kang, Xudong ;
Huang, Yuwen ;
Wang, Kuikui ;
Yang, Gongping .
IEEE GEOSCIENCE AND REMOTE SENSING LETTERS, 2022, 19
[42]   Transfer Beyond the Field of View: Dense Panoramic Semantic Segmentation via Unsupervised Domain Adaptation [J].
Zhang, Jiaming ;
Ma, Chaoxiang ;
Yang, Kailun ;
Roitberg, Alina ;
Peng, Kunyu ;
Stiefelhagen, Rainer .
IEEE TRANSACTIONS ON INTELLIGENT TRANSPORTATION SYSTEMS, 2022, 23 (07) :9478-9491
[43]   A One-Stage Domain Adaptation Network With Image Alignment for Unsupervised Nighttime Semantic Segmentation [J].
Wu, Xinyi ;
Wu, Zhenyao ;
Ju, Lili ;
Wang, Song .
IEEE TRANSACTIONS ON PATTERN ANALYSIS AND MACHINE INTELLIGENCE, 2023, 45 (01) :58-72
[44]   Threshold-Adaptive Unsupervised Focal Loss for Domain Adaptation of Semantic Segmentation [J].
Yan, Weihao ;
Qian, Yeqiang ;
Wang, Chunxiang ;
Yang, Ming .
IEEE TRANSACTIONS ON INTELLIGENT TRANSPORTATION SYSTEMS, 2023, 24 (01) :752-763
[45]   BiFDANet: Unsupervised Bidirectional Domain Adaptation for Semantic Segmentation of Remote Sensing Images [J].
Cai, Yuxiang ;
Yang, Yingchun ;
Zheng, Qiyi ;
Shen, Zhengwei ;
Shang, Yongheng ;
Yin, Jianwei ;
Shi, Zhongtian .
REMOTE SENSING, 2022, 14 (01)
[46]   Cluster Alignment With Target Knowledge Mining for Unsupervised Domain Adaptation Semantic Segmentation [J].
Wang, Shuang ;
Zhao, Dong ;
Zhang, Chi ;
Guo, Yuwei ;
Zang, Qi ;
Gu, Yu ;
Li, Yi ;
Jiao, Licheng .
IEEE TRANSACTIONS ON IMAGE PROCESSING, 2022, 31 :7403-7418
[47]   PANDA: A Polarized Attention Network for Enhanced Unsupervised Domain Adaptation in Semantic Segmentation [J].
Kao, Chiao-Wen ;
Chang, Wei-Ling ;
Lee, Chun-Chieh ;
Fan, Kuo-Chin .
ELECTRONICS, 2024, 13 (21)
[48]   Survey on Unsupervised Domain Adaptation for Semantic Segmentation for Visual Perception in Automated Driving [J].
Schwonberg, Manuel ;
Niemeijer, Joshua ;
Termohlen, Jan-Aike ;
Schafer, Jorg P. ;
Schmidt, Nico M. ;
Gottschalk, Hanno ;
Fingscheidt, Tim .
IEEE ACCESS, 2023, 11 :54296-54336
[49]   Co-training with Credal Models [J].
Soullard, Yann ;
Destercke, Sebastien ;
Thouvenin, Indira .
ARTIFICIAL NEURAL NETWORKS IN PATTERN RECOGNITION, 2016, 9896 :92-104
[50]   PDA: Progressive Domain Adaptation for Semantic Segmentation [J].
Liao, Muxin ;
Tian, Shishun ;
Zhang, Yuhang ;
Hua, Guoguang ;
Zou, Wenbin ;
Li, Xia .
KNOWLEDGE-BASED SYSTEMS, 2024, 284