Unsupervised Domain Adaptation for Semantic Segmentation with Task Disentanglement and Consistency Learning

Unsupervised domain adaptation (UDA) is a prevalent solution to save manual annotation costs for semantic segmentation by transferring beneficial knowledge from a labeled source domain to an unlabeled target domain. However, existing UDA methods typically integrate domain alignment and semantic segmentation tasks within a single backbone, resulting in excessive coupling between tasks and interfering with their learning. Besides, the pixel-level annotations of the source domain provide stronger supervision, causing model bias towards the source domain. To address these issues, we propose a UDA method via task disentanglement and consistency learning for semantic segmentation. Specifically, two Siamese shallow feature extractors are constructed to respectively extract source and target shallow features. The source shallow features are imposed with adversarial constraint to achieve domain alignment, while the target features are directly utilized for semantic segmentation. In this way, the two tasks are effectively disentangled into different shallow feature extractors, greatly reducing inter-task interference. Based on the well-aligned knowledge, overfitting to the source domain is also alleviated, and the semantic segmentation model focuses more on the target domain. Additionally, to further mitigate the model bias problem and explore the discriminative information in the target domain, we propose a consistency learning strategy with uncertainty estimation to constrain the prediction consistency of two differently initialized segmentation networks, thus enhancing the adaptability of the model to unlabeled target domain data and promoting robust and reliable knowledge extractions. Experimental results on the GTA5 -> Cityscapes task have demonstrated the superiority of our method compared to other state-of-the-art approaches.