CGFNet: Cross-Guided Fusion Network for RGB-T Salient Object Detection

被引:136
作者
Wang, Jie [1 ,2 ]
Song, Kechen [1 ,2 ]
Bao, Yanqi [1 ,2 ]
Huang, Liming [1 ,2 ]
Yan, Yunhui [1 ,2 ]
机构
[1] Northeastern Univ, Sch Mech Engn & Automat, Shenyang 110819, Liaoning, Peoples R China
[2] Northeastern Univ, Key Lab Vibrat & Control Prop Syst, Minist Educ China, Shenyang 110819, Liaoning, Peoples R China
基金
中国国家自然科学基金;
关键词
Feature extraction; Decoding; Object detection; Semantics; Image edge detection; Task analysis; Image segmentation; Salient object detection; RGB-T; cross-guided fusion; cross-level enhancement;
D O I
10.1109/TCSVT.2021.3099120
中图分类号
TM [电工技术]; TN [电子技术、通信技术];
学科分类号
0808 ; 0809 ;
摘要
RGB salient object detection (SOD) has made great progress. However, the performance of this single-modal salient object detection will be significantly decreased when encountering some challenging scenes, such as low light or darkness. To deal with the above challenges, thermal infrared (T) image is introduced into the salient object detection. This fused modal is called RGB-T salient object detection. To achieve deep mining of the unique characteristics of single modal and the full integration of cross-modality information, a novel Cross-Guided Fusion Network (CGFNet) for RGB-T salient object detection is proposed. Specifically, a Cross-Scale Alternate Guiding Fusion (CSAGF) module is proposed to mine the high-level semantic information and provide global context support. Subsequently, we design a Guidance Fusion Module (GFM) to achieve sufficient cross-modality fusion by using single modal as the main guidance and the other modal as auxiliary. Finally, the Cross-Guided Fusion Module (CGFM) is presented and serves as the main decoding block. And each decoding block is consists of two parts with two modalities information of each being the main guidance, i.e., cross-shared Cross-Level Enhancement (CLE) and Global Auxiliary Enhancement (GAE). The main difference between the two parts is that the GFM using different modalities as the main guide. The comprehensive experimental results prove that our method achieves better performance than the state-of-the-art salient detection methods. The source code has released at: https://github.com/wangjie0825/CGFNet.git.
引用
收藏
页码:2949 / 2961
页数:13
相关论文
共 68 条
[1]   Triplet-Graph Reasoning Network for Few-Shot Metal Generic Surface Defect Segmentation [J].
Bao, Yanqi ;
Song, Kechen ;
Liu, Jie ;
Wang, Yanyan ;
Yan, Yunhui ;
Yu, Han ;
Li, Xingjie .
IEEE TRANSACTIONS ON INSTRUMENTATION AND MEASUREMENT, 2021, 70
[2]   Three-Stream Attention-Aware Network for RGB-D Salient Object Detection [J].
Chen, Hao ;
Li, Youfu .
IEEE TRANSACTIONS ON IMAGE PROCESSING, 2019, 28 (06) :2825-2835
[3]   Multi-modal fusion network with multi-scale multi-path and cross-modal interactions for RGB-D salient object detection [J].
Chen, Hao ;
Li, Youfu ;
Su, Dan .
PATTERN RECOGNITION, 2019, 86 :376-385
[4]   DeepLab: Semantic Image Segmentation with Deep Convolutional Nets, Atrous Convolution, and Fully Connected CRFs [J].
Chen, Liang-Chieh ;
Papandreou, George ;
Kokkinos, Iasonas ;
Murphy, Kevin ;
Yuille, Alan L. .
IEEE TRANSACTIONS ON PATTERN ANALYSIS AND MACHINE INTELLIGENCE, 2018, 40 (04) :834-848
[5]  
Chen Q, 2021, AAAI CONF ARTIF INTE, V35, P1063
[6]   DPANet: Depth Potentiality-Aware Gated Attention Network for RGB-D Salient Object Detection [J].
Chen, Zuyao ;
Cong, Runmin ;
Xu, Qianqian ;
Huang, Qingming .
IEEE TRANSACTIONS ON IMAGE PROCESSING, 2021, 30 :7012-7024
[7]  
Chen ZY, 2020, AAAI CONF ARTIF INTE, V34, P10599
[8]  
Cheng Y, 2014, IEEE INT CON MULTI
[9]   Going From RGB to RGBD Saliency: A Depth-Guided Transformation Model [J].
Cong, Runmin ;
Lei, Jianjun ;
Fu, Huazhu ;
Hou, Junhui ;
Huang, Qingming ;
Kwong, Sam .
IEEE TRANSACTIONS ON CYBERNETICS, 2020, 50 (08) :3627-3639
[10]   Review of Visual Saliency Detection With Comprehensive Information [J].
Cong, Runmin ;
Lei, Jianjun ;
Fu, Huazhu ;
Cheng, Ming-Ming ;
Lin, Weisi ;
Huang, Qingming .
IEEE TRANSACTIONS ON CIRCUITS AND SYSTEMS FOR VIDEO TECHNOLOGY, 2019, 29 (10) :2941-2959