MULTI-STAGE FEATURE ALIGNMENT NETWORK FOR VIDEO SUPER-RESOLUTION

被引:0
作者
Suzuki, Keito [1 ]
Ikehara, Masaaki [1 ]
机构
[1] Keio Univ, EEE Dept, Yokohama, Kanagawa 2238522, Japan
来源
2022 IEEE INTERNATIONAL CONFERENCE ON IMAGE PROCESSING, ICIP | 2022年
关键词
Video Super-Resolution; Deep Learning; Convolutional Neural Networks;
D O I
10.1109/ICIP46576.2022.9897627
中图分类号
TP18 [人工智能理论];
学科分类号
081104 ; 0812 ; 0835 ; 1405 ;
摘要
Video super-resolution aims at generating high-resolution video frames using multiple adjacent low-resolution frames. An important aspect of video super-resolution is the alignment of neighboring frames to the reference frame. Previous methods directly align the frames either using optical flowor deformable convolution. However, directly estimating the motion from low-resolution inputs is hard since they often contain blur and noise that hinder the image quality. To address this problem, we propose to conduct feature alignment across multiple stages to more accurately align the frames. Furthermore, to fuse the aligned features, we introduce a novel Attentional Feature Fusion Block that applies a spatial attention mechanism to avoid areas with occlusion or misalignment. Experimental results show that the proposed method achieves competitive performance to other state-of-the-art super-resolution methods while reducing the network parameters.
引用
收藏
页码:2001 / 2005
页数:5
相关论文
共 24 条
[1]   Fast, Accurate, and Lightweight Super-Resolution with Cascading Residual Network [J].
Ahn, Namhyuk ;
Kang, Byungkon ;
Sohn, Kyung-Ah .
COMPUTER VISION - ECCV 2018, PT X, 2018, 11214 :256-272
[2]  
Baker S., 1999, SUPER RESOLUTION OPT
[3]  
Bascle B., 1996, Computer Vision - ECCV '96. 4th Eurpean Conference on Computer Proceedings, P573
[4]   Real-Time Video Super-Resolution with Spatio-Temporal Networks and Motion Compensation [J].
Caballero, Jose ;
Ledig, Christian ;
Aitken, Andrew ;
Acosta, Alejandro ;
Totz, Johannes ;
Wang, Zehan ;
Shi, Wenzhe .
30TH IEEE CONFERENCE ON COMPUTER VISION AND PATTERN RECOGNITION (CVPR 2017), 2017, :2848-2857
[5]   Design, Analysis and Testing of a Component Head Injury Criterion Tester for Pilot [J].
Cheng, Nannan ;
Wang, Liang ;
Liu, Yongguang ;
Zhang, Chunyou .
IEEE ACCESS, 2019, 7 :114888-114896
[6]   Deformable Convolutional Networks [J].
Dai, Jifeng ;
Qi, Haozhi ;
Xiong, Yuwen ;
Li, Yi ;
Zhang, Guodong ;
Hu, Han ;
Wei, Yichen .
2017 IEEE INTERNATIONAL CONFERENCE ON COMPUTER VISION (ICCV), 2017, :764-773
[7]   Joint MAP registration and high-resolution image estimation using a sequence of undersampled images [J].
Hardie, RC ;
Barnard, KJ ;
Armstrong, EE .
IEEE TRANSACTIONS ON IMAGE PROCESSING, 1997, 6 (12) :1621-1633
[8]   Lightweight Image Super-Resolution with Information Multi-distillation Network [J].
Hui, Zheng ;
Gao, Xinbo ;
Yang, Yunchu ;
Wang, Xiumei .
PROCEEDINGS OF THE 27TH ACM INTERNATIONAL CONFERENCE ON MULTIMEDIA (MM'19), 2019, :2024-2032
[9]   Video Super-Resolution With Convolutional Neural Networks [J].
Kappeler, Armin ;
Yoo, Seunghwan ;
Dai, Qiqin ;
Katsaggelos, Aggelos K. .
IEEE TRANSACTIONS ON COMPUTATIONAL IMAGING, 2016, 2 (02) :109-122
[10]  
Kingma DP, 2014, ADV NEUR IN, V27