LENFusion: A Joint Low-Light Enhancement and Fusion Network for Nighttime Infrared and Visible Image Fusion

被引:27
作者
Chen, Jun [1 ,2 ,3 ]
Yang, Liling [1 ,2 ,3 ]
Liu, Wei [1 ,2 ,3 ]
Tian, Xin [4 ]
Ma, Jiayi [4 ]
机构
[1] China Univ Geosci, Sch Automat, Wuhan 430074, Peoples R China
[2] China Univ Geosci, Hubei Key Lab Adv Control & Intelligent Automat Co, Wuhan 430074, Peoples R China
[3] Minist Educ, Engn Res Ctr Intelligent Technol Geoexplorat, Wuhan 430074, Peoples R China
[4] Wuhan Univ, Elect Informat Sch, Wuhan 430072, Peoples R China
基金
中国国家自然科学基金;
关键词
Dual-attention mechanism; image fusion; infrared; low-light enhancement; nighttime; PERFORMANCE; NEST;
D O I
10.1109/TIM.2024.3390194
中图分类号
TM [电工技术]; TN [电子技术、通信技术];
学科分类号
0808 ; 0809 ;
摘要
Contemporary image fusion methods face challenges in meeting the demands of dim nighttime environments, often accompanied by the concealment of image details in dark regions. In this article, we introduce a novel approach, named LENFusion, which achieves a beneficial interaction between low-light enhancement and image fusion in the form of a feedback loop. LENFusion is primarily divided into three components: luminance adjustment network (LAN), re-enhancement and fusion network (RFN), and luminance feedback network (LFN). The enhancement is performed in two stages. In the initial stage, LAN applies adaptive luminance adjustment to the original visible image. Subsequently, RFN achieves secondary enhancement and feature fusion with a clever combination of dual-attention mechanism, which motivates the fusion results to have high contrast and sharpness. Finally, LFN uses the luminance feedback loss to guide the luminance information of the fused images back to the LAN, effectively avoiding inappropriate enhancement of the images that do not meet the fusion requirements. In addition, we propose a reference-free color loss method for nighttime image fusion. Extensive comparison and generalization experiments have verified the superior fusion performance of LANFusion. Our code is publicly available at: https://github.com/Liling-yang/LENFsuion.
引用
收藏
页码:1 / 15
页数:15
相关论文
共 49 条
[1]   Fast Gradient-Based Algorithms for Constrained Total Variation Image Denoising and Deblurring Problems [J].
Beck, Amir ;
Teboulle, Marc .
IEEE TRANSACTIONS ON IMAGE PROCESSING, 2009, 18 (11) :2419-2434
[2]   Learning to See in the Dark [J].
Chen, Chen ;
Chen, Qifeng ;
Xu, Jia ;
Koltun, Vladlen .
2018 IEEE/CVF CONFERENCE ON COMPUTER VISION AND PATTERN RECOGNITION (CVPR), 2018, :3291-3300
[3]   Detail preserved fusion of visible and infrared images using regional saliency extraction and multi-scale image decomposition [J].
Cui, Guangmang ;
Feng, Huajun ;
Xu, Zhihai ;
Li, Qi ;
Chen, Yueting .
OPTICS COMMUNICATIONS, 2015, 341 :199-209
[4]   Image quality measures and their performance [J].
Eskicioglu, AM ;
Fisher, PS .
IEEE TRANSACTIONS ON COMMUNICATIONS, 1995, 43 (12) :2959-2965
[5]   Learning a Simple Low-light Image Enhancer from Paired Low-light Instances [J].
Fu, Zhenqi ;
Yang, Yan ;
Tu, Xiaotong ;
Huang, Yue ;
Ding, Xinghao ;
Ma, Kai-Kuang .
2023 IEEE/CVF CONFERENCE ON COMPUTER VISION AND PATTERN RECOGNITION (CVPR), 2023, :22252-22261
[6]   Zero-Reference Deep Curve Estimation for Low-Light Image Enhancement [J].
Guo, Chunle ;
Li, Chongyi ;
Guo, Jichang ;
Loy, Chen Change ;
Hou, Junhui ;
Kwong, Sam ;
Cong, Runmin .
2020 IEEE/CVF CONFERENCE ON COMPUTER VISION AND PATTERN RECOGNITION (CVPR), 2020, :1777-1786
[7]   A new image fusion performance metric based on visual information fidelity [J].
Han, Yu ;
Cai, Yunze ;
Cao, Yin ;
Xu, Xiaoming .
INFORMATION FUSION, 2013, 14 (02) :127-135
[8]  
Hwang S, 2015, PROC CVPR IEEE, P1037, DOI 10.1109/CVPR.2015.7298706
[9]   LLVIP: A Visible-infrared Paired Dataset for Low-light Vision [J].
Jia, Xinyu ;
Zhu, Chuang ;
Li, Minzhen ;
Tang, Wenqi ;
Zhou, Wenli .
2021 IEEE/CVF INTERNATIONAL CONFERENCE ON COMPUTER VISION WORKSHOPS (ICCVW 2021), 2021, :3489-3497
[10]   EnlightenGAN: Deep Light Enhancement Without Paired Supervision [J].
Jiang, Yifan ;
Gong, Xinyu ;
Liu, Ding ;
Cheng, Yu ;
Fang, Chen ;
Shen, Xiaohui ;
Yang, Jianchao ;
Zhou, Pan ;
Wang, Zhangyang .
IEEE TRANSACTIONS ON IMAGE PROCESSING, 2021, 30 :2340-2349