Multi-Scale Feature Fusion with Attention Mechanism Based on CGAN Network for Infrared Image Colorization

被引:4
|
作者
Ai, Yibo [1 ,2 ]
Liu, Xiaoxi [1 ]
Zhai, Haoyang [1 ]
Li, Jie [3 ]
Liu, Shuangli [4 ]
An, Huilong [3 ]
Zhang, Weidong [1 ]
机构
[1] Univ Sci & Technol Beijing, Natl Ctr Mat Serv Safety, Beijing 100083, Peoples R China
[2] Southern Marine Sci & Engn Guangdong Lab Zhuhai, Zhuhai 519082, Peoples R China
[3] HBIS Mat Inst, 385 South Sports St, Shijiazhuang 050023, Peoples R China
[4] Hesteel Grp Tangsteel Co, 9 Binhe Rd, Tangshan 063000, Peoples R China
来源
APPLIED SCIENCES-BASEL | 2023年 / 13卷 / 08期
关键词
attention mechanism module; Generative Adversarial Network (GAN); image colorization; infrared images; multi-scale feature fusion;
D O I
10.3390/app13084686
中图分类号
O6 [化学];
学科分类号
0703 ;
摘要
This paper proposes a colorization algorithm for infrared images based on a Conditional Generative Adversarial Network (CGAN) with multi-scale feature fusion and attention mechanisms, aiming to address issues such as color leakage and unclear semantics in existing infrared image coloring methods. Firstly, we improved the generator of the CGAN network by incorporating a multi-scale feature extraction module into the U-Net architecture to fuse features from different scales, thereby enhancing the network's ability to extract features and improving its semantic understanding, which improves the problems of color leakage and blurriness during colorization. Secondly, we enhanced the discriminator of the CGAN network by introducing an attention mechanism module, which includes channel attention and spatial attention modules, to better distinguish between real and generated images, thereby improving the semantic clarity of the resulting infrared images. Finally, we jointly improved the generator and discriminator of the CGAN network by incorporating both the multi-scale feature fusion module and attention mechanism module. We tested our method on a dataset containing both infrared and near-infrared images, which retains more detailed features while also preserving the advantages of existing infrared images. The experimental results show that our proposed method achieved a peak signal-to-noise ratio (PSNR) of 16.5342 dB and a structural similarity index (SSIM) of 0.6385 on an RGB-NIR (Red, Green, Blue-Near Infrared) testing dataset, representing a 5% and 13% improvement over the original CGAN network, respectively. These results demonstrate the effectiveness of our proposed algorithm in addressing the issues of color leakage and unclear semantics in the original network. The proposed method in this paper is not only applicable to infrared image colorization but can also be widely applied to the colorization of remote sensing and CT images.
引用
收藏
页数:17
相关论文
共 50 条
  • [21] Multi-scale Convolutional Feature Fusion Network Based on Attention Mechanism for IoT Traffic Classification
    Niandong Liao
    Jiayu Guan
    International Journal of Computational Intelligence Systems, 17
  • [22] Multi-scale Convolutional Feature Fusion Network Based on Attention Mechanism for IoT Traffic Classification
    Liao, Niandong
    Guan, Jiayu
    INTERNATIONAL JOURNAL OF COMPUTATIONAL INTELLIGENCE SYSTEMS, 2024, 17 (01)
  • [23] Infrared and visual image fusion based on multi-scale feature decomposition
    Yan, Huibin
    Li, Zhongmin
    OPTIK, 2020, 203
  • [24] Multi-scale high and low feature fusion attention network for intestinal image classification
    Li, Sheng
    Zhu, Beibei
    Guo, Xinran
    Ye, Shufang
    Ye, Jietong
    Zhuang, Yongwei
    He, Xiongxiong
    SIGNAL IMAGE AND VIDEO PROCESSING, 2023, 17 (06) : 2877 - 2886
  • [25] Collaborative Attention Guided Multi-Scale Feature Fusion Network for Medical Image Segmentation
    Xu, Zhenghua
    Tian, Biao
    Liu, Shijie
    Wang, Xiangtao
    Yuan, Di
    Gu, Junhua
    Chen, Junyang
    Lukasiewicz, Thomas
    Leung, Victor C. M.
    IEEE TRANSACTIONS ON NETWORK SCIENCE AND ENGINEERING, 2024, 11 (02): : 1857 - 1871
  • [26] Multi-scale high and low feature fusion attention network for intestinal image classification
    Sheng Li
    Beibei Zhu
    Xinran Guo
    Shufang Ye
    Jietong Ye
    Yongwei Zhuang
    Xiongxiong He
    Signal, Image and Video Processing, 2023, 17 : 2877 - 2886
  • [27] Siamese Network with Multi-scale Feature Fusion and Dual Attention Mechanism for Template Matching
    Zhao, Kai
    He, Binbing
    Pan, Shiju
    Zhu, Yuan
    2022 41ST CHINESE CONTROL CONFERENCE (CCC), 2022, : 6588 - 6592
  • [28] AM-MSFF: A Pest Recognition Network Based on Attention Mechanism and Multi-Scale Feature Fusion
    Zhang, Meng
    Yang, Wenzhong
    Chen, Danny
    Fu, Chenghao
    Wei, Fuyuan
    ENTROPY, 2024, 26 (05)
  • [29] MEEAFusion: Multi-Scale Edge Enhancement and Joint Attention Mechanism Based Infrared and Visible Image Fusion
    Xie, Yingjiang
    Fei, Zhennan
    Deng, Da
    Meng, Lingshuai
    Niu, Fu
    Sun, Jinggong
    SENSORS, 2024, 24 (17)
  • [30] Facial Expression Recognition Based on Multi-scale Feature Fusion Convolutional Neural Network and Attention Mechanism
    Wu, Yana
    Jia, Kebin
    Sun, Zhonghua
    PATTERN RECOGNITION AND COMPUTER VISION, PRCV 2021, PT II, 2021, 13020 : 324 - 335