Pedestrian detection-driven cascade network for infrared and visible image fusion

Infrared and visible image fusion aims to generate a single fused image, which not only contains rich texture details, but also beneficial for high-level vision tasks. However, the existing fusion methods tend to focus on visual quality and statistical metrics while ignoring the connection between fusion results and high-level visual tasks. In order to improve the pedestrian detection performance of the fused image and retaining pixel-level information, we propose a novel two-stage pedestrian detection-driven cascade network. In the first stage, we propose a dual-branch autoencoder network that utilizes spatial feature alignment module (SFAM) to integrate complementary information. In the second stage, we cascade the fusion module with pedestrian detection task to guide the fusion process. Compared with nine algorithms on two public datasets, experimental results show that the proposed network generates fused images with higher metrics and better visual perception. Furthermore, our method outperforms in terms of pedestrian detection accuracy on two pretrained classical object detection networks.