Bidirectional feature learning network for RGB-D salient object detection

被引:2
作者
Niu, Ye
Zhou, Sanping [1 ]
Dong, Yonghao
Wang, Le
Wang, Jinjun
Zheng, Nanning
机构
[1] Xi An Jiao Tong Univ, Inst Artificial Intelligence & Robot, Xian, Peoples R China
基金
中国博士后科学基金; 国家重点研发计划;
关键词
RGB-D salient object detection; Bidirectional feature fusion; Dual consistency loss; IMAGE; FUSION;
D O I
10.1016/j.patcog.2024.110304
中图分类号
TP18 [人工智能理论];
学科分类号
081104 ; 0812 ; 0835 ; 1405 ;
摘要
RGB-D salient object detection aims to perform the pixel-wise localization of salient objects from both RGB and depth images, whose challenge mainly comes from how to learn complementary features from each modality. Existing works often use increasingly large models for performance enhancement, which need large memory and time consumption in practice. In this paper, we propose a simple yet effective Bidirectional Feature Learning Network (BFLNet) for RGB-D salient object detection under limited memory and time conditions. To achieve accurate performance with lightweight backbone networks, an effective Bidirectional Feature Fusion (BFF) module is designed to merge features from both RGB and depth streams, in which the crossmodal fusions and cross-scale fusions are jointly conducted to fuse the immediate features in multiple scales and multiple modals. What is more, a simple Dual Consistency Loss (DCL) function is designed to prompt cross -modal fusion by keeping the consistency between cross -modal target predictions. Extensive experiments on four benchmark datasets demonstrate that our method has achieved the state-of-the-art performance with high efficiency in RGB-D salient object detection. Code will be available at https://github.com/nightskynostar/BFLNet.
引用
收藏
页数:9
相关论文
共 45 条
[1]   Cross-modal hierarchical interaction network for RGB-D salient object detection [J].
Bi, Hongbo ;
Wu, Ranwan ;
Liu, Ziqi ;
Zhu, Huihui ;
Zhang, Cong ;
Xiang, Tian -Zhu .
PATTERN RECOGNITION, 2023, 136
[2]   Progressively Complementarity-aware Fusion Network for RGB-D Salient Object Detection [J].
Chen, Hao ;
Li, Youfu .
2018 IEEE/CVF CONFERENCE ON COMPUTER VISION AND PATTERN RECOGNITION (CVPR), 2018, :3051-3060
[3]   3-D Convolutional Neural Networks for RGB-D Salient Object Detection and Beyond [J].
Chen, Qian ;
Zhang, Zhenxi ;
Lu, Yanye ;
Fu, Keren ;
Zhao, Qijun .
IEEE TRANSACTIONS ON NEURAL NETWORKS AND LEARNING SYSTEMS, 2024, 35 (03) :4309-4323
[4]   Xception: Deep Learning with Depthwise Separable Convolutions [J].
Chollet, Francois .
30TH IEEE CONFERENCE ON COMPUTER VISION AND PATTERN RECOGNITION (CVPR 2017), 2017, :1800-1807
[5]  
Deng-Ping Fan, 2020, Computer Vision - ECCV 2020. 16th European Conference. Proceedings. Lecture Notes in Computer Science (LNCS 12357), P275, DOI 10.1007/978-3-030-58610-2_17
[6]   Depth really Matters: Improving Visual Salient Region Detection with Depth [J].
Desingh, Karthik ;
Krishna, K. Madhava ;
Rajan, Deepu ;
Jawahar, C. V. .
PROCEEDINGS OF THE BRITISH MACHINE VISION CONFERENCE 2013, 2013,
[7]   Rethinking RGB-D Salient Object Detection: Models, Data Sets, and Large-Scale Benchmarks [J].
Fan, Deng-Ping ;
Lin, Zheng ;
Zhang, Zhao ;
Zhu, Menglong ;
Cheng, Ming-Ming .
IEEE TRANSACTIONS ON NEURAL NETWORKS AND LEARNING SYSTEMS, 2021, 32 (05) :2075-2089
[8]   Local to Global Feature Learning for Salient Object Detection [J].
Feng, Xuelu ;
Zhou, Sanping ;
Zhu, Zixin ;
Wang, Le ;
Hua, Gang .
PATTERN RECOGNITION LETTERS, 2022, 162 :81-88
[9]   Siamese Network for RGB-D Salient Object Detection and Beyond [J].
Fu, Keren ;
Fan, Deng-Ping ;
Ji, Ge-Peng ;
Zhao, Qijun ;
Shen, Jianbing ;
Zhu, Ce .
IEEE TRANSACTIONS ON PATTERN ANALYSIS AND MACHINE INTELLIGENCE, 2022, 44 (09) :5541-5559
[10]  
Howard AG, 2017, Arxiv, DOI [arXiv:1704.04861, DOI 10.48550/ARXIV.1704.04861]