HDCTfusion: Hybrid Dual-Branch Network Based on CNN and Transformer for Infrared and Visible Image Fusion

被引:0
作者
Wang, Wenqing [1 ,2 ]
Li, Lingzhou [1 ]
Yang, Yifei [1 ]
Liu, Han [1 ,2 ]
Guo, Runyuan [1 ,2 ]
机构
[1] Xian Univ Technol, Sch Automat & Informat Engn, Xian 710048, Peoples R China
[2] Xian Univ Technol, Shaanxi Key Lab Complex Syst Control & Intelligen, Xian 710048, Peoples R China
基金
中国国家自然科学基金;
关键词
image fusion; infrared image; visible image; Transformer; CNN; PERFORMANCE;
D O I
10.3390/s24237729
中图分类号
O65 [分析化学];
学科分类号
070302 ; 081704 ;
摘要
The purpose of infrared and visible image fusion is to combine the advantages of both and generate a fused image that contains target information and has rich details and contrast. However, existing fusion algorithms often overlook the importance of incorporating both local and global feature extraction, leading to missing key information in the fused image. To address these challenges, this paper proposes a dual-branch fusion network combining convolutional neural network (CNN) and Transformer, which enhances the feature extraction capability and motivates the fused image to contain more information. Firstly, a local feature extraction module with CNN as the core is constructed. Specifically, the residual gradient module is used to enhance the ability of the network to extract texture information. Also, jump links and coordinate attention are used in order to relate shallow features to deeper ones. In addition, a global feature extraction module based on Transformer is constructed. Through the powerful ability of Transformer, the global context information of the image can be captured and the global features are fully extracted. The effectiveness of the proposed method in this paper is verified on different experimental datasets, and it is better than most of the current advanced fusion algorithms.
引用
收藏
页数:20
相关论文
共 50 条
  • [31] DHCT-GAN: Improving EEG Signal Quality with a Dual-Branch Hybrid CNN-Transformer Network
    Cai, Yinan
    Meng, Zhao
    Huang, Dian
    SENSORS, 2025, 25 (01)
  • [32] MSFNet: MultiStage Fusion Network for infrared and visible image fusion
    Wang, Chenwu
    Wu, Junsheng
    Zhu, Zhixiang
    Chen, Hao
    NEUROCOMPUTING, 2022, 507 : 26 - 39
  • [33] DCFusion: A Dual-Frequency Cross-Enhanced Fusion Network for Infrared and Visible Image Fusion
    Wu, Dan
    Han, Mina
    Yang, Yang
    Zhao, Shan
    Rao, Yujing
    Li, Hao
    Lin, Xing
    Zhou, Chengjiang
    Bai, Haicheng
    IEEE TRANSACTIONS ON INSTRUMENTATION AND MEASUREMENT, 2023, 72
  • [34] MATCNN: Infrared and Visible Image Fusion Method Based on Multiscale CNN With Attention Transformer
    Liu, Jingjing
    Zhang, Li
    Zeng, Xiaoyang
    Liu, Wanquan
    Zhang, Jianhua
    IEEE TRANSACTIONS ON INSTRUMENTATION AND MEASUREMENT, 2025, 74
  • [35] A CNN- and Transformer-Based Dual-Branch Network for Change Detection with Cross-Layer Feature Fusion and Edge Constraints
    Wang, Xiaofeng
    Guo, Zhongyu
    Feng, Ruyi
    REMOTE SENSING, 2024, 16 (14)
  • [36] ITFuse: An interactive transformer for infrared and visible image fusion
    Tang, Wei
    He, Fazhi
    Liu, Yu
    PATTERN RECOGNITION, 2024, 156
  • [37] CTFusion: CNN-transformer-based self-supervised learning for infrared and visible image fusion
    Du, Keying
    Fang, Liuyang
    Chen, Jie
    Chen, Dongdong
    Lai, Hua
    Mathematical Biosciences and Engineering, 2024, 21 (07) : 6710 - 6730
  • [38] Cross-UNet: dual-branch infrared and visible image fusion framework based on cross-convolution and attention mechanism
    Wang, Xuejiao
    Hua, Zhen
    Li, Jinjiang
    VISUAL COMPUTER, 2023, 39 (10) : 4801 - 4818
  • [39] CGTF: Convolution-Guided Transformer for Infrared and Visible Image Fusion
    Li, Jing
    Zhu, Jianming
    Li, Chang
    Chen, Xun
    Yang, Bin
    IEEE TRANSACTIONS ON INSTRUMENTATION AND MEASUREMENT, 2022, 71
  • [40] RITFusion: Reinforced Interactive Transformer Network for Infrared and Visible Image Fusion
    Li, Xiaoling
    Li, Yanfeng
    Chen, Houjin
    Peng, Yahui
    Chen, Luyifu
    Wang, Minjun
    IEEE TRANSACTIONS ON INSTRUMENTATION AND MEASUREMENT, 2024, 73