Depth Guidance Unsupervised Domain Adaptation for Semantic Segmentation

被引：0
|
作者：
Lu J. [1 ]
Shi J. [1 ]
Zhu H. [1 ]
Sun Y. [2 ]
Cheng Z. [3 ]
机构：
[1] School of Comрuter Science and Engineering, Jiangsu University of Science and Technology, Zhenjiang
[2] State Key Laboratory for Novel Software Technology, Nanjing University, Nanjing
[3] School of Software Technology, Zhejiang University, Hangzhou
来源：
Jisuanji Fuzhu Sheji Yu Tuxingxue Xuebao/Journal of Computer-Aided Design and Computer Graphics | 2024年 / 36卷 / 01期
关键词：
depth estimation; multi-task learning; semantic segmentation; unsupervised domain adaptation;
D O I：
10.3724/SP.J.1089.2024.19824
中图分类号：
学科分类号：
摘要：
To improve the segmentation performance and solve the problem of poor generalization of the model in different data domains, we propose a method based on depth information for semantic segmentation in the context of unsupervised domain adaptation. It includes a Depth-aware Adaptation Framework(DAF) and a Intra-domain Adaptation(IDA) strategy. Firstly, DAF is proposed to adapt domains by capitalizing on the inherent correlations of semantic and depth information. Then a novel light-weight depth estimation network is designed provide additional depth information, and we fuse semantic and depth information by cross-task interaction, then align the distribution in depth-aware space between source and target domains. Finally, IDA strategy is proposed to bridge the distribution gap inside the target domain. To this end, a depth-aware ranking strategy is presented to separate target domain into sub-source domain and sub-target domain, and then we perform the alignment between sub-source domain and sub-target domain. Experiments on SYNTHIA-2-Cityscapes and SYNTHIA-2-Mapillary cross-domain tasks show that our method achieves significant improvement(46.7% mIoU and 73.3% mIoU, respectively) compared with the similar methods. © 2024 Institute of Computing Technology. All rights reserved.
引用
收藏
页码：133 / 141
页数：8
相关论文
共 29 条
  • [1] Feng D, Haase-Schutz C, Rosenbaum L, Et al., Deep multi-modal object detection and semantic segmentation for autonomous driving: Datasets, methods, and challenges, IEEE Transactions on Intelligent Transportation Systems, 22, 3, pp. 1341-1360, (2021)
  • [2] Khan K, Khan R U, Ahmad K, Et al., Face segmentation: a journey from classical to deep learning paradigm, approaches, trends, and directions, IEEE Access, 8, pp. 58683-58699, (2020)
  • [3] Lee K H, Ros G, Li J, Et al., SPIGAN: privileged adversarial learning from simulation
  • [4] Vu T H, Jain H, Bucher M, Et al., ADVENT: adversarial entropy minimization for domain adaptation in semantic segmentation, Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 2512-2521, (2019)
  • [5] Hoffman J, Wang D Q, Yu F, Et al., FCNs in the wild: pixel-level adversarial and constraint-based adaptation
  • [6] Tsai Y H, Sohn K, Schulter S, Et al., Domain adaptation for structured output via discriminative patch representations, Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 1456-1465, (2019)
  • [7] Zheng Z D, Yang Y., Rectifying pseudo label learning via uncertainty estimation for domain adaptive semantic segmentation, International Journal of Computer Vision, 129, 4, pp. 1106-1120, (2021)
  • [8] Vu T H, Jain H, Bucher M, Et al., DADA: depth-aware domain adaptation in semantic segmentation, Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 7363-7372, (2019)
  • [9] Saha S, Obukhov A, Paudel D P, Et al., Learning to relate depth and semantics for unsupervised domain adaptation, Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 8193-8203, (2021)
  • [10] Wang Q, Dai D X, Hoyer L, Et al., Domain adaptive semantic segmentation with self-supervised depth estimation, Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 8495-8505, (2021)
123