Enhanced Feature Alignment for Unsupervised Domain Adaptation of Semantic Segmentation

被引:26
作者
Chen, Tao [1 ]
Wang, Shui-Hua [2 ]
Wang, Qiong [1 ]
Zhang, Zheng [3 ,4 ]
Xie, Guo-Sen [1 ]
Tang, Zhenmin [1 ]
机构
[1] Nanjing Univ Sci & Technol, Sch Comp Sci & Engn, Nanjing 210094, Peoples R China
[2] Univ Leicester, Sch Math & Actuarial Sci, Leicester LE1 7RH, Leics, England
[3] Harbin Inst Technol, ShenzhenKey Lab Visual Object Detect & Recognit, Shenzhen 518055, Peoples R China
[4] Peng Cheng Lab, Shenzhen 518055, Peoples R China
基金
中国国家自然科学基金;
关键词
Adversarial learning; Domain adaptation; pseudo label; semantic segmentation; NETWORK;
D O I
10.1109/TMM.2021.3106095
中图分类号
TP [自动化技术、计算机技术];
学科分类号
0812 ;
摘要
Unsupervised domain adaptation for semantic segmentation aims to transfer knowledge from a labeled source domain to another unlabeled target domain. However, due to the label noise and domain mismatch, learning directly from source domain data tends to have poor performance. Though adversarial learning methods strive to reduce domain discrepancies by aligning feature distributions, traditional methods suffer from the training imbalance and feature distortion problems. Besides, due to the absence of target domain labels, the classifier is blind to features from the target domain during training. Consequently, the final classifier overfits the source domain features and usually fails to predict the structured outputs of the target domain. To alleviate these problems, we focus on enhancing the adversarial learning based feature alignment from three perspectives. First, a classification constrained discriminator is proposed to balance the adversarial training and alleviate the feature distortion problem. Next, to alleviate the classifier overfitting problem, self-training is collaboratively used to learn a domain robust classifier with target domain pseudo labels. Moreover, an efficient class centroid calculation module is proposed and the domain discrepancy is further reduced by aligning the feature centroids of the same class from different domains. Experimental evaluations on GTA5 -> Cityscapes and SYNTHIA -> Cityscapes demonstrate state-of-the-art results compared to other counterpart methods. The source code and models have been made available at.(1)
引用
收藏
页码:1042 / 1054
页数:13
相关论文
共 66 条
[1]  
[Anonymous], 2015, PROC INT C LEARNING
[2]   SegNet: A Deep Convolutional Encoder-Decoder Architecture for Image Segmentation [J].
Badrinarayanan, Vijay ;
Kendall, Alex ;
Cipolla, Roberto .
IEEE TRANSACTIONS ON PATTERN ANALYSIS AND MACHINE INTELLIGENCE, 2017, 39 (12) :2481-2495
[3]   Progressive Feature Alignment for Unsupervised Domain Adaptation [J].
Chen, Chaoqi ;
Xie, Weiping ;
Huang, Wenbing ;
Rong, Yu ;
Ding, Xinghao ;
Huang, Yue ;
Xu, Tingyang ;
Huang, Junzhou .
2019 IEEE/CVF CONFERENCE ON COMPUTER VISION AND PATTERN RECOGNITION (CVPR 2019), 2019, :627-636
[4]   Encoder-Decoder with Atrous Separable Convolution for Semantic Image Segmentation [J].
Chen, Liang-Chieh ;
Zhu, Yukun ;
Papandreou, George ;
Schroff, Florian ;
Adam, Hartwig .
COMPUTER VISION - ECCV 2018, PT VII, 2018, 11211 :833-851
[5]   DeepLab: Semantic Image Segmentation with Deep Convolutional Nets, Atrous Convolution, and Fully Connected CRFs [J].
Chen, Liang-Chieh ;
Papandreou, George ;
Kokkinos, Iasonas ;
Murphy, Kevin ;
Yuille, Alan L. .
IEEE TRANSACTIONS ON PATTERN ANALYSIS AND MACHINE INTELLIGENCE, 2018, 40 (04) :834-848
[6]  
Chen M., 2011, Advances in neural information processing systems, P2456
[7]   Domain Adaptation for Semantic Segmentation with Maximum Squares Loss [J].
Chen, Minghao ;
Xue, Hongyang ;
Cai, Deng .
2019 IEEE/CVF INTERNATIONAL CONFERENCE ON COMPUTER VISION (ICCV 2019), 2019, :2090-2099
[8]   Semantically Meaningful Class Prototype Learning for One-Shot Image Segmentation [J].
Chen, Tao ;
Xie, Guo-Sen ;
Yao, Yazhou ;
Wang, Qiong ;
Shen, Fumin ;
Tang, Zhenmin ;
Zhang, Jian .
IEEE TRANSACTIONS ON MULTIMEDIA, 2022, 24 :968-980
[9]   No More Discrimination: Cross City Adaptation of Road Scene Segmenters [J].
Chen, Yi-Hsin ;
Chen, Wei-Yu ;
Chen, Yu-Ting ;
Tsai, Bo-Cheng ;
Wang, Yu-Chiang Frank ;
Sun, Min .
2017 IEEE INTERNATIONAL CONFERENCE ON COMPUTER VISION (ICCV), 2017, :2011-2020
[10]   ROAD: Reality Oriented Adaptation for Semantic Segmentation of Urban Scenes [J].
Chen, Yuhua ;
Li, Wen ;
Van Gool, Luc .
2018 IEEE/CVF CONFERENCE ON COMPUTER VISION AND PATTERN RECOGNITION (CVPR), 2018, :7892-7901