A review on infrared and visible image fusion algorithms based on neural networks

被引:13
作者
Yang, Kaixuan [1 ,2 ,3 ,4 ]
Xiang, Wei [1 ,2 ]
Chen, Zhenshuai [1 ,2 ,3 ,4 ]
Zhang, Jian [1 ,2 ,3 ,4 ]
Liu, Yunpeng [1 ,2 ]
机构
[1] Chinese Acad Sci, Key Lab Optoelect Informat Proc, Shenyang 110016, Peoples R China
[2] Chinese Acad Sci, Shenyang Inst Automat, Shenyang 110016, Peoples R China
[3] Chinese Acad Sci, Inst Robot & Intelligent Mfg, Shenyang 110169, Peoples R China
[4] Univ Chinese Acad Sci, Beijing 100049, Peoples R China
关键词
Image fusion; Neural network; Infrared and visible images; Image processing; GENERATIVE ADVERSARIAL NETWORK; INFORMATION; NEST;
D O I
10.1016/j.jvcir.2024.104179
中图分类号
TP [自动化技术、计算机技术];
学科分类号
0812 ;
摘要
Infrared and visible image fusion represents a significant segment within the image fusion domain. The recent surge in image processing hardware advancements, including GPUs, TPUs, and cloud computing platforms, has facilitated the fusion of extensive datasets from multiple sensors. Given the remarkable proficiency of neural networks in image feature extraction and fusion, their application in infrared and visible image fusion has emerged as a prominent research area in recent years. This article begins by providing an overview of the current mainstream algorithms for infrared and visible image fusion based on neural networks, detailing the principles of various image fusion algorithms, their representative works, and their respective advantages and disadvantages. Subsequently, it introduces domain -relevant datasets, evaluation metrics, and some typical application scenarios. Finally, the article conducts qualitative and quantitative evaluations of the fusion results of various state-of-the-art algorithms and offers future research prospects based on experimental results.
引用
收藏
页数:25
相关论文
共 162 条
[1]   A Robust and Automated Vision-Based Human Fall Detection System Using 3D Multi-Stream CNNs with an Image Fusion Technique [J].
Alanazi, Thamer ;
Babutain, Khalid ;
Muhammad, Ghulam .
APPLIED SCIENCES-BASEL, 2023, 13 (12)
[2]   Efficient thermal face recognition method using optimized curvelet features for biometric authentication [J].
Ali, Mona A. S. ;
Eltoukhy, Mohamed Meselhy ;
Rajeena, P. P. Fathimathul ;
Gaber, Tarek .
PLOS ONE, 2023, 18 (06)
[3]  
An Xiaodong, 2022, Computer Engineering and Applications, P64, DOI 10.3778/j.issn.1002-8331.2203-0600
[4]  
Arjovsky M., 2017, arXiv, DOI 10.48550/arXiv.1701.04862
[5]  
Arora S, 2017, PR MACH LEARN RES, V70
[6]   A new image quality metric for image fusion: The sum of the correlations of differences [J].
Aslantas, V. ;
Bendes, E. .
AEU-INTERNATIONAL JOURNAL OF ELECTRONICS AND COMMUNICATIONS, 2015, 69 (12) :160-166
[7]  
Bino T., 2023, Modality correlation aware sparse representation and kiou tracker for rgb-infrared videos
[8]   Physiologically motivated image fusion using pulse coupled neural networks [J].
Broussard, RP ;
Rogers, SK .
APPLICATIONS AND SCIENCE OF ARTIFICIAL NEURAL NETWORKS II, 1996, 2760 :372-383
[9]   End-to-End Object Detection with Transformers [J].
Carion, Nicolas ;
Massa, Francisco ;
Synnaeve, Gabriel ;
Usunier, Nicolas ;
Kirillov, Alexander ;
Zagoruyko, Sergey .
COMPUTER VISION - ECCV 2020, PT I, 2020, 12346 :213-229
[10]   CrossViT: Cross-Attention Multi-Scale Vision Transformer for Image Classification [J].
Chen, Chun-Fu ;
Fan, Quanfu ;
Panda, Rameswar .
2021 IEEE/CVF INTERNATIONAL CONFERENCE ON COMPUTER VISION (ICCV 2021), 2021, :347-356