Infrared and visible image fusion method based on principal component analysis network and multi-scale morphological gradient

In infrared (IR) and visible image fusion, energy conservation and detail extraction are two key problems. We propose a novel IR and visible image fusion model based on principal component analysis network (PCANet) and multi-scale morphological gradient (MSMG), aiming to better preserve energy and extract details. Firstly, we obtain the features of IR and visible images through PCANet. The features obtained by PCANet have more powerful representation capabilities in IR target perception and visible detail description. Secondly, we develop a fusion strategy combining MSMG with guided filter to obtain the corresponding feature map weights. This fusion strategy is able to preserve more IR image energy and extract more visible image details. Finally, a weighted-averaging strategy is used to obtain the fused image. The effectiveness of the proposed method is verified by two datasets with more than 80 pairs of source images in total. Compared with 17 representative methods, the experimental results demonstrate that the proposed method can achieve the state-of-the-art in both visual quality and objective evaluation.