Relation Matters: Foreground-Aware Graph-Based Relational Reasoning for Domain Adaptive Object Detection

被引:26
作者
Chen, Chaoqi [1 ]
Li, Jiongcheng [2 ]
Zhou, Hong-Yu [1 ]
Han, Xiaoguang [3 ]
Huang, Yue [2 ]
Ding, Xinghao [2 ]
Yu, Yizhou [1 ]
机构
[1] Univ Hong Kong, Dept Comp Sci, Hong Kong, Peoples R China
[2] Xiamen Univ, Dept Informat & Commun Engn, Xiamen 361005, Peoples R China
[3] Chinese Univ Hong Kong Shenzhen, Shenzhen Res Inst Big Data, Shenzhen 518172, Peoples R China
基金
中国国家自然科学基金;
关键词
Cognition; Feature extraction; Semantics; Object detection; Training; Task analysis; Knowledge transfer; Domain adaptive object detection; foreground-aware; relational reasoning; graph structure; intra- and inter-domain; ALIGNMENT; KERNEL;
D O I
10.1109/TPAMI.2022.3179445
中图分类号
TP18 [人工智能理论];
学科分类号
081104 ; 0812 ; 0835 ; 1405 ;
摘要
Domain Adaptive Object Detection (DAOD) focuses on improving the generalization ability of object detectors via knowledge transfer. Recent advances in DAOD strive to change the emphasis of the adaptation process from global to local in virtue of fine-grained feature alignment methods. However, both the global and local alignment approaches fail to capture the topological relations among different foreground objects as the explicit dependencies and interactions between and within domains are neglected. In this case, only seeking one-vs-one alignment does not necessarily ensure the precise knowledge transfer. Moreover, conventional alignment-based approaches may be vulnerable to catastrophic overfitting regarding those less transferable regions (e.g., backgrounds) due to the accumulation of inaccurate localization results in the target domain. To remedy these issues, we first formulate DAOD as an open-set domain adaptation problem, in which the foregrounds and backgrounds are seen as the "known classes" and "unknown class" respectively. Accordingly, we propose a new and general framework for DAOD, named Foreground-aware Graph-based Relational Reasoning (FGRR), which incorporates graph structures into the detection pipeline to explicitly model the intra- and inter-domain foreground object relations on both pixel and semantic spaces, thereby endowing the DAOD model with the capability of relational reasoning beyond the popular alignment-based paradigm. FGRR first identifies the foreground pixels and regions by searching reliable correspondence and cross-domain similarity regularization respectively. The inter-domain visual and semantic correlations are hierarchically modeled via bipartite graph structures, and the intra-domain relations are encoded via graph attention mechanisms. Through message-passing, each node aggregates semantic and contextual information from the same and opposite domain to substantially enhance its expressive power. Empirical results demonstrate that the proposed FGRR exceeds the state-of-the-art performance on four DAOD benchmarks.
引用
收藏
页码:3677 / 3694
页数:18
相关论文
共 103 条
[51]  
Long MS, 2018, ADV NEUR IN, V31
[52]  
Long MS, 2015, PR MACH LEARN RES, V37, P97
[53]  
Luo Y., 2020, PMLR, P6468
[54]   Adversarial Bipartite Graph Learning for Video Domain Adaptation [J].
Luo, Yadan ;
Huang, Zi ;
Wang, Zijian ;
Zhang, Zheng ;
Baktashmotlagh, Mahsa .
MM '20: PROCEEDINGS OF THE 28TH ACM INTERNATIONAL CONFERENCE ON MULTIMEDIA, 2020, :19-27
[55]   GCAN: Graph Convolutional Adversarial Network for Unsupervised Domain Adaptation [J].
Ma, Xinhong ;
Zhang, Tianzhu ;
Xu, Changsheng .
2019 IEEE/CVF CONFERENCE ON COMPUTER VISION AND PATTERN RECOGNITION (CVPR 2019), 2019, :8258-8268
[56]   INbreast: Toward a Full-field Digital Mammographic Database [J].
Moreira, Ines C. ;
Amaral, Igor ;
Domingues, Ines ;
Cardoso, Antonio ;
Cardoso, Maria Joao ;
Cardoso, Jaime S. .
ACADEMIC RADIOLOGY, 2012, 19 (02) :236-248
[57]   Unsupervised Domain Adaptation of Object Detectors: A Survey [J].
Oza, Poojan ;
Sindagi, Vishwanath A. ;
Vibashan, V. S. ;
Patel, Vishal M. .
IEEE TRANSACTIONS ON PATTERN ANALYSIS AND MACHINE INTELLIGENCE, 2024, 46 (06) :4018-4040
[58]   A Survey on Transfer Learning [J].
Pan, Sinno Jialin ;
Yang, Qiang .
IEEE TRANSACTIONS ON KNOWLEDGE AND DATA ENGINEERING, 2010, 22 (10) :1345-1359
[59]  
Pan YW, 2020, PROC CVPR IEEE, P13864, DOI 10.1109/CVPR42600.2020.01388
[60]   Moment Matching for Multi-Source Domain Adaptation [J].
Peng, Xingchao ;
Bai, Qinxun ;
Xia, Xide ;
Huang, Zijun ;
Saenko, Kate ;
Wang, Bo .
2019 IEEE/CVF INTERNATIONAL CONFERENCE ON COMPUTER VISION (ICCV 2019), 2019, :1406-1415