FECFusion: Infrared and visible image fusion network based on fast edge convolution

被引:8
作者
Chen, Zhaoyu [1 ]
Fan, Hongbo [2 ]
Ma, Meiyan [1 ]
Shao, Dangguo [1 ,3 ]
机构
[1] Kunming Univ Sci & Technol, Fac Informat Engn & Automat, Kunming 650500, Peoples R China
[2] Kunming Univ Sci & Technol, Fac Modern Agr Engn, Kunming 650500, Peoples R China
[3] Kunming Univ Sci & Technol, Yunnan Prov Key Lab Comp, Kunming 650500, Peoples R China
基金
中国国家自然科学基金;
关键词
image fusion; edge operator; structural re-parameterization; infrared and visible images; deep learning; PERFORMANCE; FRAMEWORK; NEST;
D O I
10.3934/mbe.2023717
中图分类号
Q [生物科学];
学科分类号
07 ; 0710 ; 09 ;
摘要
The purpose of infrared and visible image fusion is to integrate the complementary infor-mation from heterogeneous images in order to enhance their detailed scene information. However, existing deep learning fusion methods suffer from an imbalance between fusion performance and com-putational resource consumption. Additionally, fusion layers or fusion rules fail to effectively combine heteromodal feature information. To address these challenges, this paper presents a novel algorithm called infrared and visible image fusion network base on fast edge convolution (FECFusion). During the training phase, the proposed algorithm enhances the extraction of texture features in the source image through the utilization of structural re-parameterization edge convolution (RECB) with embed-ded edge operators. Subsequently, the attention fusion module (AFM) is employed to sufficiently fuze both unique and public information from the heteromodal features. In the inference stage, we further optimize the training network using the structural reparameterization technique, resulting in a VGG-like network architecture. This optimization improves the fusion speed while maintaining the fusion performance. To evaluate the performance of the proposed FECFusion algorithm, qualitative and quantitative experiments are conducted. Seven advanced fusion algorithms are compared using MSRS, TNO, and M3FD datasets. The results demonstrate that the fusion algorithm presented in this paper achieves superior performance in multiple evaluation metrics, while consuming fewer computa-tional resources. Consequently, the proposed algorithm yields better visual results and provides richer scene detail information.
引用
收藏
页码:16060 / 16082
页数:23
相关论文
共 50 条
[1]  
Alexander T., 2014, TNO Image Fusion Dataset
[2]   A new image quality metric for image fusion: The sum of the correlations of differences [J].
Aslantas, V. ;
Bendes, E. .
AEU-INTERNATIONAL JOURNAL OF ELECTRONICS AND COMMUNICATIONS, 2015, 69 (12) :160-166
[3]   Multi-Focus Image Fusion Based on Multi-Scale Gradients and Image Matting [J].
Chen, Jun ;
Li, Xuejiao ;
Luo, Linbo ;
Ma, Jiayi .
IEEE TRANSACTIONS ON MULTIMEDIA, 2022, 24 :655-667
[4]   Infrared and visible image fusion based on target-enhanced multiscale transform decomposition [J].
Chen, Jun ;
Li, Xuejiao ;
Luo, Linbo ;
Mei, Xiaoguang ;
Ma, Jiayi .
INFORMATION SCIENCES, 2020, 508 :64-78
[5]   Encoder-Decoder with Atrous Separable Convolution for Semantic Image Segmentation [J].
Chen, Liang-Chieh ;
Zhu, Yukun ;
Papandreou, George ;
Schroff, Florian ;
Adam, Hartwig .
COMPUTER VISION - ECCV 2018, PT VII, 2018, 11211 :833-851
[6]   The Cityscapes Dataset for Semantic Urban Scene Understanding [J].
Cordts, Marius ;
Omran, Mohamed ;
Ramos, Sebastian ;
Rehfeld, Timo ;
Enzweiler, Markus ;
Benenson, Rodrigo ;
Franke, Uwe ;
Roth, Stefan ;
Schiele, Bernt .
2016 IEEE CONFERENCE ON COMPUTER VISION AND PATTERN RECOGNITION (CVPR), 2016, :3213-3223
[7]   Scaling Up Your Kernels to 31x31: Revisiting Large Kernel Design in CNNs [J].
Ding, Xiaohan ;
Zhang, Xiangyu ;
Han, Jungong ;
Ding, Guiguang .
2022 IEEE/CVF CONFERENCE ON COMPUTER VISION AND PATTERN RECOGNITION (CVPR), 2022, :11953-11965
[8]   Diverse Branch Block: Building a Convolution as an Inception-like Unit [J].
Ding, Xiaohan ;
Zhang, Xiangyu ;
Han, Jungong ;
Ding, Guiguang .
2021 IEEE/CVF CONFERENCE ON COMPUTER VISION AND PATTERN RECOGNITION, CVPR 2021, 2021, :10881-10890
[9]   RepVGG: Making VGG-style ConvNets Great Again [J].
Ding, Xiaohan ;
Zhang, Xiangyu ;
Ma, Ningning ;
Han, Jungong ;
Ding, Guiguang ;
Sun, Jian .
2021 IEEE/CVF CONFERENCE ON COMPUTER VISION AND PATTERN RECOGNITION, CVPR 2021, 2021, :13728-13737
[10]   ACNet: Strengthening the Kernel Skeletons for Powerful CNN via Asymmetric Convolution Blocks [J].
Ding, Xiaohan ;
Guo, Yuchen ;
Ding, Guiguang ;
Han, Jungong .
2019 IEEE/CVF INTERNATIONAL CONFERENCE ON COMPUTER VISION (ICCV 2019), 2019, :1911-1920