CDNet: Complementary Depth Network for RGB-D Salient Object Detection

被引:136
作者
Jin, Wen-Da [1 ]
Xu, Jun [2 ]
Han, Qi [3 ]
Zhang, Yi [1 ]
Cheng, Ming-Ming [3 ]
机构
[1] Tianjin Univ, Coll Intelligence & Comp, Tianjin 300350, Peoples R China
[2] Nankai Univ, Sch Stat & Data Sci, Tianjin 300371, Peoples R China
[3] Nankai Univ, Coll Comp Sci, TKLNDST, Tianjin 300350, Peoples R China
基金
中国国家自然科学基金;
关键词
Feature extraction; Ions; Fuses; Task analysis; Object detection; Streaming media; Predictive models; RGB-D salient object detection; depth estimation; cross-modal feature fusion; FUSION;
D O I
10.1109/TIP.2021.3060167
中图分类号
TP18 [人工智能理论];
学科分类号
081104 ; 0812 ; 0835 ; 1405 ;
摘要
Current RGB-D salient object detection (SOD) methods utilize the depth stream as complementary information to the RGB stream. However, the depth maps are usually of low-quality in existing RGB-D SOD datasets. Most RGB-D SOD networks trained with these datasets would produce error-prone results. In this paper, we propose a novel Complementary Depth Network (CDNet) to well exploit saliency-informative depth features for RGB-D SOD. To alleviate the influence of low-quality depth maps to RGB-D SOD, we propose to select saliency-informative depth maps as the training targets and leverage RGB features to estimate meaningful depth maps. Besides, to learn robust depth features for accurate prediction, we propose a new dynamic scheme to fuse the depth features extracted from the original and estimated depth maps with adaptive weights. What's more, we design a two-stage cross-modal feature fusion scheme to well integrate the depth features with the RGB ones, further improving the performance of our CDNet on RGB-D SOD. Experiments on seven benchmark datasets demonstrate that our CDNet outperforms state-of-the-art RGB-D SOD methods. The code is publicly available at https://github.com/blanclist/CDNet.
引用
收藏
页码:3376 / 3390
页数:15
相关论文
共 68 条
[1]  
Achanta R, 2009, PROC CVPR IEEE, P1597, DOI 10.1109/CVPRW.2009.5206596
[2]   Salient Object Detection: A Benchmark [J].
Borji, Ali ;
Sihite, Dicky N. ;
Itti, Laurent .
COMPUTER VISION - ECCV 2012, PT II, 2012, 7573 :414-429
[3]   Improved Saliency Detection in RGB-D Images Using Two-Phase Depth Estimation and Selective Deep Fusion [J].
Chen, Chenglizhao ;
Wei, Jipeng ;
Peng, Chong ;
Zhang, Weizhong ;
Qin, Hong .
IEEE TRANSACTIONS ON IMAGE PROCESSING, 2020, 29 :4296-4307
[4]   Three-Stream Attention-Aware Network for RGB-D Salient Object Detection [J].
Chen, Hao ;
Li, Youfu .
IEEE TRANSACTIONS ON IMAGE PROCESSING, 2019, 28 (06) :2825-2835
[5]   RGBD Salient Object Detection via Disentangled Cross-Modal Fusion [J].
Chen, Hao ;
Deng, Yongjian ;
Li, Youfu ;
Hung, Tzu-Yi ;
Lin, Guosheng .
IEEE TRANSACTIONS ON IMAGE PROCESSING, 2020, 29 (29) :8407-8416
[6]   Progressively Complementarity-aware Fusion Network for RGB-D Salient Object Detection [J].
Chen, Hao ;
Li, Youfu .
2018 IEEE/CVF CONFERENCE ON COMPUTER VISION AND PATTERN RECOGNITION (CVPR), 2018, :3051-3060
[7]   Multi-modal fusion network with multi-scale multi-path and cross-modal interactions for RGB-D salient object detection [J].
Chen, Hao ;
Li, Youfu ;
Su, Dan .
PATTERN RECOGNITION, 2019, 86 :376-385
[8]   PREATTENTIVE CO-SALIENCY DETECTION [J].
Chen, Hwann-Tzong .
2010 IEEE INTERNATIONAL CONFERENCE ON IMAGE PROCESSING, 2010, :1117-1120
[9]   Global Contrast based Salient Region Detection [J].
Cheng, Ming-Ming ;
Zhang, Guo-Xin ;
Mitra, Niloy J. ;
Huang, Xiaolei ;
Hu, Shi-Min .
2011 IEEE CONFERENCE ON COMPUTER VISION AND PATTERN RECOGNITION (CVPR), 2011, :409-416
[10]  
Cheng Y., 2014, ICIMCS, P23