A dual-branch infrared and visible image fusion network using progressive image-wise feature transfer

被引:0
作者
Xu, Shaoping [1 ]
Zhou, Changfei [1 ]
Xiao, Jian [1 ]
Tao, Wuyong [1 ]
Dai, Tianyu [1 ]
机构
[1] Nanchang Univ, Sch Math & Comp Sci, Nanchang 330031, Jiangxi, Peoples R China
基金
中国国家自然科学基金;
关键词
Infrared and visible image fusion; Dual-branch fusion network; Progressive image-wise feature transfer; Transformer module; CLIP loss; NEST;
D O I
10.1016/j.jvcir.2024.104190
中图分类号
TP [自动化技术、计算机技术];
学科分类号
0812 ;
摘要
To achieve a fused image that contains rich texture details and prominent targets, we present a progressive dual-branch infrared and visible image fusion network called PDFusion, which incorporates the Transformer module. Initially, the proposed network is divided into two branches to extract infrared and visible features independently. Subsequently, the image-wise transfer block (ITB) is introduced to fuse the infrared and visible features at different layers, facilitating the exchange of information between features. The fused features are then fed back into both pathways to contribute to the subsequent feature extraction process. Moreover, in addition to conventional pixel-level and structured loss functions, the contrastive language- image pretraining (CLIP) loss is introduced to guide the network training. Experimental results on publicly available datasets demonstrate the promising performance of PDFusion in the task of infrared and visible image fusion. The exceptional fusion performance of the proposed fusion network can be attributed to the following reasons: (1) The ITB block, particularly with the integration of the Transformer, enhances the capability of representation learning. The Transformer module captures long-range dependencies among image features, enabling a global receptive field that integrates contextual information from the entire image. This leads to a more comprehensive fusion of features. (2) The feature loss based on the CLIP image encoder minimizes the discrepancy between the generated and target images. Consequently, it promotes the generation of semantically coherent and visually appealing fused images. The source code of our method can be found at https://github.com/Changfei-Zhou/PDFusion.
引用
收藏
页数:11
相关论文
共 50 条
  • [1] DBIF: Dual-Branch Feature Extraction Network for Infrared and Visible Image Fusion
    Zhang, Haozhe
    Cui, Rongpu
    Zheng, Zhuohang
    Gao, Shaobing
    PATTERN RECOGNITION AND COMPUTER VISION, PRCV 2024, PT VIII, 2025, 15038 : 309 - 323
  • [2] MDAN: Multilevel dual-branch attention network for infrared and visible image fusion
    Wang, Jiawei
    Jiang, Min
    Kong, Jun
    OPTICS AND LASERS IN ENGINEERING, 2024, 176
  • [3] Infrared and visible image fusion using a generative adversarial network with a dual-branch generator and matched dense blocks
    Guo, Li
    Tang, Dandan
    SIGNAL IMAGE AND VIDEO PROCESSING, 2023, 17 (05) : 1811 - 1819
  • [4] Infrared and visible image fusion using a generative adversarial network with a dual-branch generator and matched dense blocks
    Li Guo
    Dandan Tang
    Signal, Image and Video Processing, 2023, 17 : 1811 - 1819
  • [5] DDRF: Dual-branch decomposition and reconstruction architecture for infrared and visible image fusion
    Zhang, Lei
    Zhou, Qiming
    Tang, Mingliang
    Ding, Xin
    Yang, Chengwei
    Wei, Chuyuan
    Zhou, Zhimiao
    OPTICS AND LASER TECHNOLOGY, 2025, 181
  • [6] DPACFuse: Dual-Branch Progressive Learning for Infrared and Visible Image Fusion with Complementary Self-Attention and Convolution
    Zhu, Huayi
    Wu, Heshan
    Wang, Xiaolong
    He, Dongmei
    Liu, Zhenbing
    Pan, Xipeng
    SENSORS, 2023, 23 (16)
  • [7] GLFuse: A Global and Local Four-Branch Feature Extraction Network for Infrared and Visible Image Fusion
    Zhao, Genping
    Hu, Zhuyong
    Feng, Silu
    Wang, Zhuowei
    Wu, Heng
    REMOTE SENSING, 2024, 16 (17)
  • [8] A Two-Branch Fusion Network for Infrared and Visible Image Fusion
    Zhang, Weihao
    Li, Zhilin
    Li, Bin
    Zhang, Mingliang
    PATTERN RECOGNITION AND COMPUTER VISION, PT IX, PRCV 2024, 2025, 15039 : 42 - 55
  • [9] TDDFusion: A Target-Driven Dual Branch Network for Infrared and Visible Image Fusion
    Lu, Siyu
    Ye, Xiangzhou
    Rao, Junmin
    Li, Fanming
    Liu, Shijian
    SENSORS, 2024, 24 (01)
  • [10] A dual-encoder network based on multi-layer feature fusion for infrared and visible image fusion
    Huang, Shuying
    Wu, Xueqiang
    Yang, Yong
    Wan, Weiguo
    Wang, Xiaozheng
    INTERNATIONAL JOURNAL OF MACHINE LEARNING AND CYBERNETICS, 2024, 15 (10) : 4511 - 4520