Deep SR-HDR: Joint Learning of Super-Resolution and High Dynamic Range Imaging for Dynamic Scenes

The visual quality of a single image captured by a digital camera usually suffers from limited spatial resolution and low dynamic range (LDR) due to sensor constraints. To address these problems, recent works have independently applied convolutional neural networks (CNNs) to super-resolution (SR) and high dynamic range (HDR) imaging and made significant improvements in visual quality. However, directly connecting SR and HDR networks is an inefficient way to enhance image quality, because these two tasks share most of the same processing steps. To this end, we propose a deep neural network for the joint task of SR and HDR imaging, termed Deep SR-HDR, which reconstructs a high-resolution (HR) HDR image from a set of differently exposed low-resolution (LR) LDR images of a dynamic scene. Specifically, we merge the shared processing steps, including feature extraction and alignment of these two tasks. In particular, to handle large-scale complex motions, we design a multi-scale deformable module (MSDM) that estimates the sampling location offsets in a coarse-to-fine manner and then flexibly integrates useful information to compensate for the missing content in the motion regions. Then, we divide the fusion stage into two branches for HDR generation and high-frequency information extraction. With the cooperation and interactions of these modules, the proposed network reconstructs high-quality HR HDR images. Extensive qualitative and quantitative experimental results demonstrate the superiority and high efficiency of the proposed network.