FECFusion: Infrared and visible image fusion network based on fast edge convolution

被引:8
作者
Chen, Zhaoyu [1 ]
Fan, Hongbo [2 ]
Ma, Meiyan [1 ]
Shao, Dangguo [1 ,3 ]
机构
[1] Kunming Univ Sci & Technol, Fac Informat Engn & Automat, Kunming 650500, Peoples R China
[2] Kunming Univ Sci & Technol, Fac Modern Agr Engn, Kunming 650500, Peoples R China
[3] Kunming Univ Sci & Technol, Yunnan Prov Key Lab Comp, Kunming 650500, Peoples R China
基金
中国国家自然科学基金;
关键词
image fusion; edge operator; structural re-parameterization; infrared and visible images; deep learning; PERFORMANCE; FRAMEWORK; NEST;
D O I
10.3934/mbe.2023717
中图分类号
Q [生物科学];
学科分类号
07 ; 0710 ; 09 ;
摘要
The purpose of infrared and visible image fusion is to integrate the complementary infor-mation from heterogeneous images in order to enhance their detailed scene information. However, existing deep learning fusion methods suffer from an imbalance between fusion performance and com-putational resource consumption. Additionally, fusion layers or fusion rules fail to effectively combine heteromodal feature information. To address these challenges, this paper presents a novel algorithm called infrared and visible image fusion network base on fast edge convolution (FECFusion). During the training phase, the proposed algorithm enhances the extraction of texture features in the source image through the utilization of structural re-parameterization edge convolution (RECB) with embed-ded edge operators. Subsequently, the attention fusion module (AFM) is employed to sufficiently fuze both unique and public information from the heteromodal features. In the inference stage, we further optimize the training network using the structural reparameterization technique, resulting in a VGG-like network architecture. This optimization improves the fusion speed while maintaining the fusion performance. To evaluate the performance of the proposed FECFusion algorithm, qualitative and quantitative experiments are conducted. Seven advanced fusion algorithms are compared using MSRS, TNO, and M3FD datasets. The results demonstrate that the fusion algorithm presented in this paper achieves superior performance in multiple evaluation metrics, while consuming fewer computa-tional resources. Consequently, the proposed algorithm yields better visual results and provides richer scene detail information.
引用
收藏
页码:16060 / 16082
页数:23
相关论文
共 50 条
[11]   GF-Detection: Fusion with GAN of Infrared and Visible Images for Vehicle Detection at Nighttime [J].
Gao, Peng ;
Tian, Tian ;
Zhao, Tianming ;
Li, Linfeng ;
Zhang, Nan ;
Tian, Jinwen .
REMOTE SENSING, 2022, 14 (12)
[12]   A new image fusion performance metric based on visual information fidelity [J].
Han, Yu ;
Cai, Yunze ;
Cao, Yin ;
Xu, Xiaoming .
INFORMATION FUSION, 2013, 14 (02) :127-135
[13]   Current advances and future perspectives of image fusion: A comprehensive review [J].
Karim, Shahid ;
Tong, Geng ;
Li, Jinyang ;
Qadir, Akeel ;
Farooq, Umar ;
Yu, Yiting .
INFORMATION FUSION, 2023, 90 :185-217
[14]   Different Input Resolutions and Arbitrary Output Resolution: A Meta Learning-Based Deep Framework for Infrared and Visible Image Fusion [J].
Li, Huafeng ;
Cen, Yueliang ;
Liu, Yu ;
Chen, Xun ;
Yu, Zhengtao .
IEEE TRANSACTIONS ON IMAGE PROCESSING, 2021, 30 :4070-4083
[15]   RFN-Nest: An end-to-end residual fusion network for infrared and visible images [J].
Li, Hui ;
Wu, Xiao-Jun ;
Kittler, Josef .
INFORMATION FUSION, 2021, 73 :72-86
[16]   NestFuse: An Infrared and Visible Image Fusion Architecture Based on Nest Connection and Spatial/Channel Attention Models [J].
Li, Hui ;
Wu, Xiao-Jun ;
Durrani, Tariq .
IEEE TRANSACTIONS ON INSTRUMENTATION AND MEASUREMENT, 2020, 69 (12) :9645-9656
[17]   DenseFuse: A Fusion Approach to Infrared and Visible Images [J].
Li, Hui ;
Wu, Xiao-Jun .
IEEE TRANSACTIONS ON IMAGE PROCESSING, 2019, 28 (05) :2614-2623
[18]   Infrared-visible image fusion method based on sparse and prior joint saliency detection and LatLRR-FPDE [J].
Li, Yonghua ;
Liu, Gang ;
Bavirisetti, Durga Prasad ;
Gu, Xinjie ;
Zhou, Xihong .
DIGITAL SIGNAL PROCESSING, 2023, 134
[19]   Adaptive Payload Distribution in Multiple Images Steganography Based on Image Texture Features [J].
Liao, Xin ;
Yin, Jiaojiao ;
Chen, Mingliang ;
Qin, Zheng .
IEEE TRANSACTIONS ON DEPENDABLE AND SECURE COMPUTING, 2022, 19 (02) :897-911
[20]   A New Payload Partition Strategy in Color Image Steganography [J].
Liao, Xin ;
Yu, Yingbo ;
Li, Bin ;
Li, Zhongpeng ;
Qin, Zheng .
IEEE TRANSACTIONS ON CIRCUITS AND SYSTEMS FOR VIDEO TECHNOLOGY, 2020, 30 (03) :685-696