Pedestrian detection-driven cascade network for infrared and visible image fusion

被引:2
作者
Zheng, Bowen [1 ]
Huo, Hongtao [1 ]
Liu, Xiaowen [1 ]
Pang, Shan [1 ,2 ]
Li, Jing [3 ]
机构
[1] Peoples Publ Secur Univ China, Sch Informat Technol & Cyber Secur, Beijing 100038, Peoples R China
[2] Fujian Police Coll, Dept Forens Sci, Fuzhou 350007, Peoples R China
[3] Cent Univ Finance & Econ, Sch Informat, Beijing 100081, Peoples R China
基金
中国国家自然科学基金;
关键词
Infrared image; Visible image; Image fusion; Pedestrian detection;
D O I
10.1016/j.sigpro.2024.109620
中图分类号
TM [电工技术]; TN [电子技术、通信技术];
学科分类号
0808 ; 0809 ;
摘要
Infrared and visible image fusion aims to generate a single fused image, which not only contains rich texture details, but also beneficial for high-level vision tasks. However, the existing fusion methods tend to focus on visual quality and statistical metrics while ignoring the connection between fusion results and high-level visual tasks. In order to improve the pedestrian detection performance of the fused image and retaining pixel-level information, we propose a novel two-stage pedestrian detection-driven cascade network. In the first stage, we propose a dual-branch autoencoder network that utilizes spatial feature alignment module (SFAM) to integrate complementary information. In the second stage, we cascade the fusion module with pedestrian detection task to guide the fusion process. Compared with nine algorithms on two public datasets, experimental results show that the proposed network generates fused images with higher metrics and better visual perception. Furthermore, our method outperforms in terms of pedestrian detection accuracy on two pretrained classical object detection networks.
引用
收藏
页数:11
相关论文
共 45 条
[1]   A new image quality metric for image fusion: The sum of the correlations of differences [J].
Aslantas, V. ;
Bendes, E. .
AEU-INTERNATIONAL JOURNAL OF ELECTRONICS AND COMMUNICATIONS, 2015, 69 (12) :160-166
[2]  
Bavirisetti DP, 2017, 2017 20TH INTERNATIONAL CONFERENCE ON INFORMATION FUSION (FUSION), P701
[3]   Infrared and visible image fusion based on target-enhanced multiscale transform decomposition [J].
Chen, Jun ;
Li, Xuejiao ;
Luo, Linbo ;
Mei, Xiaoguang ;
Ma, Jiayi .
INFORMATION SCIENCES, 2020, 508 :64-78
[4]   Deep Convolutional Neural Network for Multi-Modal Image Restoration and Fusion [J].
Deng, Xin ;
Dragotti, Pier Luigi .
IEEE TRANSACTIONS ON PATTERN ANALYSIS AND MACHINE INTELLIGENCE, 2021, 43 (10) :3333-3348
[5]  
Deshmukh M., 2010, Int. J. Image Process., V4, P484
[6]   Perceptual Image Fusion Using Wavelets [J].
Hill, Paul ;
Al-Mualla, Mohammed Ebrahim ;
Bull, David .
IEEE TRANSACTIONS ON IMAGE PROCESSING, 2017, 26 (03) :1076-1088
[7]   Scope of validity of PSNR in image/video quality assessment [J].
Huynh-Thu, Q. ;
Ghanbari, M. .
ELECTRONICS LETTERS, 2008, 44 (13) :800-U35
[8]   LLVIP: A Visible-infrared Paired Dataset for Low-light Vision [J].
Jia, Xinyu ;
Zhu, Chuang ;
Li, Minzhen ;
Tang, Wenqi ;
Zhou, Wenli .
2021 IEEE/CVF INTERNATIONAL CONFERENCE ON COMPUTER VISION WORKSHOPS (ICCVW 2021), 2021, :3489-3497
[9]   RFN-Nest: An end-to-end residual fusion network for infrared and visible images [J].
Li, Hui ;
Wu, Xiao-Jun ;
Kittler, Josef .
INFORMATION FUSION, 2021, 73 :72-86
[10]  
Li H, 2018, INT C PATT RECOG, P2705, DOI 10.1109/ICPR.2018.8546006