Self-guided Transformer for Video Super-Resolution

The challenge of video super-resolution (VSR) is to leverage the long-range spatial-temporal correlation between low-resolution (LR) frames to generate high-resolution (LR) video frames. However, CNN-based video super-resolution approaches show limitations in modeling using long-range dependencies and non-local self-similarity. In this paper. For further spatio-temporal learning, we propose a novel self-guided transformer for video super-resolution (SGTVSR). In this framework, we customize a multi-headed self-attention based on offset-guided window (OGW-MSA). For each query element on a low-resolution reference frame, the OGW-MSA enjoys offset guidance to globally sample highly relevant key elements throughout the video. In addition, we propose a feature aggregation module that aggregates the favorable spatial information of adjacent frame features at different scales as a way to improve the video reconstruction quality. Comprehensive experiments show that our proposed self-guided transformer for video super-resolution outperforms the state-of-the-art (SOTA) method on several public datasets and produces good results visually.