Combining Pixel-Level and Structure-Level Adaptation for Semantic Segmentation

Domain adaptation for semantic segmentation requires pixel-level knowledge transfer from a labeled source domain to an unlabeled target domain. Existing approaches typically align the features of the source and target domains at different levels. However, they usually neglect the different adaptive complexities of different information flows within images. In this paper, we focus on combining two main information flows in semantic segmentation, ie., the pixel-level disparate information and image structure information. Specifically, we propose to combine two feature map-based prediction heads, which are thought to focus on pixel level and structure-level information, to accommodate different complexities by adjusting the attention to adaptation functions of the target domain. We then align the outputs from the two heads through a consistency regularization to realize informative complementarity. The combined prediction head further enables regularizing the distance between different pixel representations of different classes, thereby mitigating the mis-adaptation problem of similar classes. The proposed method can achieve more competitive results than current state-ofthe-art results on two publicly available benchmark datasets, ie., SYNTHIA -> Cityscapes and GTA5 -> Cityscapes.