PLUG-AND-PLAY REGISTRATION AND FUSION

We consider the problem of synthetically fusing a high-spatial, low-spectral resolution image with a low-spatial, high-spectral resolution image to achieve high spatial and spectral resolution. In practice, the images to be fused are usually misaligned and need to be registered before fusion is carried out. However, due to significant difference in spatial resolutions (between the input images), it can be difficult to register them accurately. We consider a variational framework for simultaneous registration and fusion along with regularization that is built upon a standard observation (forward) model. Using a mix of alternating minimization and proximal gradient descent, we obtain an algorithm in which we iteratively optimize over rotations/translations, the model mismatch, and the regularization term. Motivated by the "plug-and-play" paradigm for image restoration, we propose to replace (i) the alignment process by an efficient registration method, and (ii) the proximal map (of the regularizer) with a powerful denoiser. As the iterations proceed, we notice that better registration and regularization results in improved fusion. We demonstrate that our method is competitive with state-of-the-art fusion algorithms on standard datasets, and is particularly effective even for larger misalignments.