Group-based bi-directional recurrent wavelet neural network for efficient video super-resolution (VSR)

被引:4
作者
Choi, Young-Ju [1 ]
Lee, Young-Woon [2 ]
Kim, Byung-Gyu [1 ]
机构
[1] Sookmyung Womens Univ, Dept IT Engn, Seoul, South Korea
[2] Sunmoon Univ, Dept Comp Engn, Asan, South Korea
关键词
Attention mechanism; Discrete wavelet transform; Recurrent neural network; Video super-resolution;
D O I
10.1016/j.patrec.2022.11.014
中图分类号
TP18 [人工智能理论];
学科分类号
081104 ; 0812 ; 0835 ; 1405 ;
摘要
Video super-resolution (VSR) is an important technology for enhancing the quality of video frames. The recurrent neural network (RNN)-based approach is suitable for sequential data because it can use accumulated temporal information. However, since existing methods only tend to capture slow and symmetrical motion with low frame rate, there are still limitations to restore the missing details for more dynamic motion. Most of the previous methods using spatial information treat different types of the spatial features identically. It leads to lack of obtaining meaningful information and enhancing the fine details. We propose a group-based bi-directional recurrent wavelet neural network (GBR-WNN) to exploit spatio-temporal information effectively. The proposed group-based bi-directional RNN (GBR) framework is built on the well-structured process with the group of pictures (GOP). In a GOP, we resolves the low-resolution (LR) frames from border frames to center target frame. Because super-resolved features in a GOP are cumulative, neighboring features are improved progressively and asymmetrical motion can be dealt with. Also, we propose a temporal wavelet attention (TWA) adopting attention module for both spatial and temporal features simultaneously based on discrete wavelet transform. Experiments show that the proposed scheme achieves superior performance compared with state-of-the-art methods. (c) 2022 Elsevier B.V. All rights reserved.
引用
收藏
页码:246 / 253
页数:8
相关论文
共 39 条
[1]  
Ben Niu, 2020, Computer Vision - ECCV 2020. 16th European Conference. Proceedings. Lecture Notes in Computer Science (LNCS 12357), P191, DOI 10.1007/978-3-030-58610-2_12
[2]   Real-Time Video Super-Resolution with Spatio-Temporal Networks and Motion Compensation [J].
Caballero, Jose ;
Ledig, Christian ;
Aitken, Andrew ;
Acosta, Alejandro ;
Totz, Johannes ;
Wang, Zehan ;
Shi, Wenzhe .
30TH IEEE CONFERENCE ON COMPUTER VISION AND PATTERN RECOGNITION (CVPR 2017), 2017, :2848-2857
[3]  
Caelles S, 2019, Arxiv, DOI [arXiv:1905.00737, 10.48550/arXiv.1905.00737]
[4]  
Cao JZ, 2023, Arxiv, DOI arXiv:2106.06847
[5]   BasicVSR: The Search for Essential Components in Video Super-Resolution and Beyond [J].
Chan, Kelvin C. K. ;
Wang, Xintao ;
Yu, Ke ;
Dong, Chao ;
Loy, Chen Change .
2021 IEEE/CVF CONFERENCE ON COMPUTER VISION AND PATTERN RECOGNITION, CVPR 2021, 2021, :4945-4954
[6]  
Chan Kelvin CK, 2022, P IEEECVF C COMPUTER, P5972
[7]   Wavelet Attention Embedding Networks for Video Super-Resolution [J].
Choi, Young-Ju ;
Lee, Young-Woon ;
Kim, Byung-Gyu .
2020 25TH INTERNATIONAL CONFERENCE ON PATTERN RECOGNITION (ICPR), 2021, :7314-7320
[8]  
Dai QQ, 2015, IEEE IMAGE PROC, P83, DOI 10.1109/ICIP.2015.7350764
[9]   Learning a Deep Convolutional Network for Image Super-Resolution [J].
Dong, Chao ;
Loy, Chen Change ;
He, Kaiming ;
Tang, Xiaoou .
COMPUTER VISION - ECCV 2014, PT IV, 2014, 8692 :184-199
[10]  
Fan Y., 2019, P IEEECVF C COMPUTER