Dual-Attention-Based Feature Aggregation Network for Infrared and Visible Image Fusion

被引:31
作者
Tang, Zhimin [1 ,2 ,3 ]
Xiao, Guobao [1 ]
Guo, Junwen [1 ,2 ,3 ]
Wang, Shiping [2 ,3 ]
Ma, Jiayi [4 ]
机构
[1] Minjiang Univ, Coll Comp & Control Engn, Fuzhou 350108, Peoples R China
[2] Fuzhou Univ, Coll Comp & Data Sci, Fuzhou 350108, Peoples R China
[3] Fuzhou Univ, Coll Software, Fuzhou 350108, Peoples R China
[4] Wuhan Univ, Sch Elect Informat, Wuhan 430072, Peoples R China
基金
中国国家自然科学基金;
关键词
Attention mechanisms; feature aggregation; image fusion; FRAMEWORK;
D O I
10.1109/TIM.2023.3259021
中图分类号
TM [电工技术]; TN [电子技术、通信技术];
学科分类号
0808 ; 0809 ;
摘要
Infrared and visible image fusion aims to produce fused images which retain rich texture details and high pixel intensity in the source images. In this article, we propose a dual-attention-based feature aggregation network for infrared and visible image fusion. Specifically, we first design a multibranch channel-attention-based feature aggregation block (MBCA) by generating multiple branches to suppress useless features from different aspects. This block is also able to adaptively aggregate the meaningful features by exploiting the interdependencies between channel features. To gather more meaningful features during the fusion process, we further design a global-local spatial-attention-based feature aggregation block (GLSA), for progressively integrating features of source images. After that, we introduce multiscale structural similarity (MS-SSIM) as loss function to evaluate the structural differences between the fused image and the source images from multiple scales. In addition, the proposed network involves strong generalization ability since our fusion model is trained on the RoadScene dataset and tested directly on the TNO and MSRS datasets. Extensive experiments on these datasets demonstrate the superiority of our network compared with the current state-of-the-art methods. The source code will be released at https://github.com/tangjunyang/Dualattention.
引用
收藏
页数:13
相关论文
共 45 条
[1]   A new image quality metric for image fusion: The sum of the correlations of differences [J].
Aslantas, V. ;
Bendes, E. .
AEU-INTERNATIONAL JOURNAL OF ELECTRONICS AND COMMUNICATIONS, 2015, 69 (12) :160-166
[2]  
Bavirisetti DP, 2017, 2017 20TH INTERNATIONAL CONFERENCE ON INFORMATION FUSION (FUSION), P701
[3]   Two-scale image fusion of visible and infrared images using saliency detection [J].
Bavirisetti, Durga Prasad ;
Dhuli, Ravindra .
INFRARED PHYSICS & TECHNOLOGY, 2016, 76 :52-64
[4]   Learning a Neural Solver for Multiple Object Tracking [J].
Braso, Guillem ;
Leal-Taixe, Laura .
2020 IEEE/CVF CONFERENCE ON COMPUTER VISION AND PATTERN RECOGNITION (CVPR), 2020, :6246-6256
[5]  
Ding S., 2022, IEEE T INSTRUM MEAS, V71, P1
[6]  
Ha Q, 2017, IEEE INT C INT ROBOT, P5108, DOI 10.1109/IROS.2017.8206396
[7]   Novel fusion method for visible light and infrared images based on NSST-SF-PCNN [J].
Kong, Weiwei ;
Zhang, Longjun ;
Lei, Yang .
INFRARED PHYSICS & TECHNOLOGY, 2014, 65 :103-112
[8]   SiamRPN plus plus : Evolution of Siamese Visual Tracking with Very Deep Networks [J].
Li, Bo ;
Wu, Wei ;
Wang, Qiang ;
Zhang, Fangyi ;
Xing, Junliang ;
Yan, Junjie .
2019 IEEE/CVF CONFERENCE ON COMPUTER VISION AND PATTERN RECOGNITION (CVPR 2019), 2019, :4277-4286
[9]   RFN-Nest: An end-to-end residual fusion network for infrared and visible images [J].
Li, Hui ;
Wu, Xiao-Jun ;
Kittler, Josef .
INFORMATION FUSION, 2021, 73 :72-86
[10]   DenseFuse: A Fusion Approach to Infrared and Visible Images [J].
Li, Hui ;
Wu, Xiao-Jun .
IEEE TRANSACTIONS ON IMAGE PROCESSING, 2019, 28 (05) :2614-2623