A multiscale residual pyramid attention network for medical image fusion

Recently, deep learning has been widely used in the imaging field. Residual, pyramid and attention networks are proposed successively, and are extensively used because of their excellent performance. However, the performance of a single network is limited. Based on this, we propose a multiscale residual pyramid attention network (MSRPAN) for medical image fusion. Our network consists of one feature extractor, fuser and reconstructor. The feature extractor is composed of three MSRPAN blocks, which are utilized to extract multiscale features. The reconstructor consists of three convolution layers, which are used to reconstruct the fused features. In addition, we propose the feature energy ratio strategy for the feature fusion process. The proposed strategy achieved better fusion results in the experiments. Compared with the existing state-of-the-art algorithms, our algorithm achieves better performance in terms of vision quality and objective metrics.