HDCTfusion: Hybrid Dual-Branch Network Based on CNN and Transformer for Infrared and Visible Image Fusion

被引:0
作者
Wang, Wenqing [1 ,2 ]
Li, Lingzhou [1 ]
Yang, Yifei [1 ]
Liu, Han [1 ,2 ]
Guo, Runyuan [1 ,2 ]
机构
[1] Xian Univ Technol, Sch Automat & Informat Engn, Xian 710048, Peoples R China
[2] Xian Univ Technol, Shaanxi Key Lab Complex Syst Control & Intelligen, Xian 710048, Peoples R China
基金
中国国家自然科学基金;
关键词
image fusion; infrared image; visible image; Transformer; CNN; PERFORMANCE;
D O I
10.3390/s24237729
中图分类号
O65 [分析化学];
学科分类号
070302 ; 081704 ;
摘要
The purpose of infrared and visible image fusion is to combine the advantages of both and generate a fused image that contains target information and has rich details and contrast. However, existing fusion algorithms often overlook the importance of incorporating both local and global feature extraction, leading to missing key information in the fused image. To address these challenges, this paper proposes a dual-branch fusion network combining convolutional neural network (CNN) and Transformer, which enhances the feature extraction capability and motivates the fused image to contain more information. Firstly, a local feature extraction module with CNN as the core is constructed. Specifically, the residual gradient module is used to enhance the ability of the network to extract texture information. Also, jump links and coordinate attention are used in order to relate shallow features to deeper ones. In addition, a global feature extraction module based on Transformer is constructed. Through the powerful ability of Transformer, the global context information of the image can be captured and the global features are fully extracted. The effectiveness of the proposed method in this paper is verified on different experimental datasets, and it is better than most of the current advanced fusion algorithms.
引用
收藏
页数:20
相关论文
共 50 条
  • [21] Infrared and Visible Image Fusion Based on Improved Dual Path Generation Adversarial Network
    Yang, Shen
    Tian, Lifan
    Liang, Jiaming
    Huang, Zefeng
    JOURNAL OF ELECTRONICS & INFORMATION TECHNOLOGY, 2023, 45 (08) : 3012 - 3021
  • [22] A Dual-Branch Multiscale Transformer Network for Hyperspectral Image Classification
    Shi, Cuiping
    Yue, Shuheng
    Wang, Liguo
    IEEE TRANSACTIONS ON GEOSCIENCE AND REMOTE SENSING, 2024, 62 : 1 - 20
  • [23] Semantic perceptive infrared and visible image fusion Transformer
    Yang, Xin
    Huo, Hongtao
    Li, Chang
    Liu, Xiaowen
    Wang, Wenxi
    Wang, Cheng
    PATTERN RECOGNITION, 2024, 149
  • [24] TDDFusion: A Target-Driven Dual Branch Network for Infrared and Visible Image Fusion
    Lu, Siyu
    Ye, Xiangzhou
    Rao, Junmin
    Li, Fanming
    Liu, Shijian
    SENSORS, 2024, 24 (01)
  • [25] PTET: A progressive token exchanging transformer for infrared and visible image fusion
    Huang, Jun
    Chen, Ziang
    Ma, Yong
    Fan, Fan
    Tang, Linfeng
    Xiang, Xinyu
    IMAGE AND VISION COMPUTING, 2024, 144
  • [26] Dual-branch network based on transformer for texture recognition
    Liu, Yangqi
    Dong, Hao
    Wang, Guodong
    Chen, Chenglizhao
    DIGITAL SIGNAL PROCESSING, 2024, 153
  • [27] TrUNet: Dual-Branch Network by Fusing CNN and Transformer for Skin Lesion Segmentation
    Chen, Wei
    Mu, Qian
    Qi, Jie
    IEEE ACCESS, 2024, 12 : 144174 - 144185
  • [28] An Infrared and Visible Image Fusion Network Based on Res2Net and Multiscale Transformer
    Tan, Binxi
    Yang, Bin
    SENSORS, 2025, 25 (03)
  • [29] PFCFuse: A Poolformer and CNN Fusion Network for Infrared-Visible Image Fusion
    Hu, Xinyu
    Liu, Yang
    Yang, Feng
    IEEE TRANSACTIONS ON INSTRUMENTATION AND MEASUREMENT, 2024, 73
  • [30] CFIFusion: Dual-Branch Complementary Feature Injection Network for Medical Image Fusion
    Xie, Yiyuan
    Yu, Lei
    Ding, Cheng
    INTERNATIONAL JOURNAL OF IMAGING SYSTEMS AND TECHNOLOGY, 2024, 34 (04)