Attentive Cross-Modal Fusion Network for RGB-D Saliency Detection

被引:22
作者
Liu, Di [1 ]
Zhang, Kao [1 ]
Chen, Zhenzhong [1 ]
机构
[1] Wuhan Univ, Sch Remote Sensing & Informat Engn, Wuhan 430079, Peoples R China
基金
中国国家自然科学基金; 国家重点研发计划;
关键词
Object detection; Saliency detection; Feature extraction; Fuses; Visualization; Computational modeling; Semantics; Cross-modal attention; residual attention; fusion refinement network; RGB-D salient object detection; OBJECT DETECTION; MODEL; DISPARITY; FIXATION;
D O I
10.1109/TMM.2020.2991523
中图分类号
TP [自动化技术、计算机技术];
学科分类号
0812 ;
摘要
In this paper, an attentive cross-modal fusion (ACMF) network is proposed for RGB-D salient object detection. The proposed method selectively fuses features in a cross-modal manner and uses a fusion refinement module to fuse output features from different resolutions. Our attentive cross-modal fusion network is built based on residual attention. In each level of ResNet output, both the RGB and depth features are turned into an identity map and a weighted attention map. The identity map is reweighted by the attention map of the paired modality. Moreover, the lower level features with higher resolution are adopted to refine the boundary of detected targets. The entire architecture can be trained end-to-end. The proposed ACMF is compared with state-of-the-art methods on eight recent datasets. The results demonstrate that our model can achieve advanced performance on RGB-D salient object detection.
引用
收藏
页码:967 / 981
页数:15
相关论文
共 70 条
[1]  
Achanta R, 2009, PROC CVPR IEEE, P1597, DOI 10.1109/CVPRW.2009.5206596
[2]   State-of-the-Art in Visual Attention Modeling [J].
Borji, Ali ;
Itti, Laurent .
IEEE TRANSACTIONS ON PATTERN ANALYSIS AND MACHINE INTELLIGENCE, 2013, 35 (01) :185-207
[3]  
Borji A, 2012, PROC CVPR IEEE, P438, DOI 10.1109/CVPR.2012.6247706
[4]   Progressively Complementarity-aware Fusion Network for RGB-D Salient Object Detection [J].
Chen, Hao ;
Li, Youfu .
2018 IEEE/CVF CONFERENCE ON COMPUTER VISION AND PATTERN RECOGNITION (CVPR), 2018, :3051-3060
[5]   Global Contrast based Salient Region Detection [J].
Cheng, Ming-Ming ;
Zhang, Guo-Xin ;
Mitra, Niloy J. ;
Huang, Xiaolei ;
Hu, Shi-Min .
2011 IEEE CONFERENCE ON COMPUTER VISION AND PATTERN RECOGNITION (CVPR), 2011, :409-416
[6]  
Cheng Y., 2014, ICIMCS, P23
[7]   Saliency Detection for Stereoscopic Images Based on Depth Confidence Analysis and Multiple Cues Fusion [J].
Cong, Runmin ;
Lei, Jianjun ;
Zhang, Changqing ;
Huang, Qingming ;
Cao, Xiaochun ;
Hou, Chunping .
IEEE SIGNAL PROCESSING LETTERS, 2016, 23 (06) :819-823
[8]   Co-Saliency Detection for RGBD Images Based on Multi-Constraint Feature Matching and Cross Label Propagation [J].
Cong, Runmin ;
Lei, Jianjun ;
Fu, Huazhu ;
Huang, Qingming ;
Cao, Xiaochun ;
Hou, Chunping .
IEEE TRANSACTIONS ON IMAGE PROCESSING, 2018, 27 (02) :568-579
[9]   Point Cloud Saliency Detection by Local and Global Feature Fusion [J].
Ding, Xiaoying ;
Lin, Weisi ;
Chen, Zhenzhong ;
Zhang, Xinfeng .
IEEE TRANSACTIONS ON IMAGE PROCESSING, 2019, 28 (11) :5379-5393
[10]   Improving Saliency Detection Based on Modeling Photographer's Intention [J].
Ding, Xiaoying ;
Chen, Zhenzhong .
IEEE TRANSACTIONS ON MULTIMEDIA, 2019, 21 (01) :124-134