A super-resolution reconstruction method based on foreground-background separation and deblurring

Due to the limited depth of field (DOF) of the camera, the background of images captured in large aperture mode is defocused and blurry, which not only results in the loss of important information in the background but also hinders the efficient reconstruction of the background regions. Usually, the super-resolution (SR) results of large aperture images are not good. Therefore, to enhance the reconstruction quality of defocused regions in large aperture images, a foreground-background separation and deblurring super-resolution (FBSDSR) method was proposed. Based on the idea of foreground-background separation processing, the large aperture image was divided into a sharp foreground region (I-f) and a blurry background region (I-b) according to the depth information. The end-to-end iterative filter adaptive network (IFAN) was used to deblur the background region I-b, refocus and restore an all-in-focus image. Finally, the enhanced super-resolution generative adversarial networks (Real-ESRGAN) which specializes in images SR of realistic scenes was used to process the sharp all-in-focus image. The proposed method realized high-quality reconstructions of both foreground and background of large aperture images. The experimental results demonstrated that the proposed method achieved effective reconstruction of the entire large aperture images clearly and solved the limitation of existing whole image reconstruction methods' inability to reconstruct defocused regions of large aperture images. The quality and resolution of large aperture images were greatly improved.