SIE: infrared and visible image fusion based on scene information embedding

In this article, we proposed an infrared and visible image fusion method based on scene information embedding, which is to obtain an fused image with high-quality target information such as pedestrian crossing, vehicles, signs and rich background texture information. First, scene information is described, so as to effectively represent highlight salient target for infrared image and texture detail for visible image. Second, we embed this scene information into multi-scale fusion framework to fuse these information and reconstruct the desired fused image. It is worth noting that scene information is constructed by pre-trained convolutional neural network, where not only information extracted by convolutional neural network is fully utilized, but also training complexity is not increased. In datasets LLVIP, MSRS, M3FD,TNO, RoadScene and elctricity1, the superiority of our proposed algorithm SIE over state-of-the-arts is demonstrated by subjective evaluation and objective evaluation including information entropy (EN), standard deviation (SD), spatial frequency (SF), average gradient (AG), sum of the correlations of differences (SCD) and mutual Information (MI) metrics. © The Author(s), under exclusive licence to Springer Science+Business Media, LLC, part of Springer Nature 2024.