Attention-adaptive and deformable convolutional modules for dynamic scene deblurring

We investigate two aspects of network architecture design for dynamic scene deblurring: (1) Learning blur characteristics and their location in dynamic scenes, which corresponds to learning what and where to attend in the channel and spatial axes, respectively. In this regard, we design an attention-adaptive module (AAM), the innovation of which is that it adaptively determines the arrangement of channel and spatial attention modules (i.e., sequentially or in parallel). Ablation experiments verified the effectiveness of the AAM by incorporating it into existing deblurring convolutional neural network (CNN) architectures. (2) Intuitively, geometric variations are widely observed in objects in dynamic scenes because different spatial regions are blurred by different motion kernels. However, owing to the fixed geometric structures in their modules, regular CNNs fail to adapt to these variations. Accordingly, we propose a deformable convolutional module (DCM) to handle geometric variations. Preliminary experiments demonstrated that incorporating the AAM and DCM into existing deblurring models can significantly improve performance. Moreover, it was empirically verified that an encoder-decoder ResBlock network incorporating the proposed modules compares favorably with state-of-the-art methods. (C) 2020 Elsevier Inc. All rights reserved.