A Fast 4K Video Frame Interpolation Using a Hybrid Task-Based Convolutional Neural Network

被引:13
作者
Ahn, Ha-Eun [1 ,2 ]
Jeong, Jinwoo [2 ]
Kim, Je Woo [2 ]
机构
[1] Kwangwoon Univ, Dept Elect Engn, Seoul 01897, South Korea
[2] Korea Elect Technol Inst, Sungnam 13509, South Korea
来源
SYMMETRY-BASEL | 2019年 / 11卷 / 05期
关键词
frame interpolation; super-resolution; edge loss; hybrid network; high-resolution image processing;
D O I
10.3390/sym11050619
中图分类号
O [数理科学和化学]; P [天文学、地球科学]; Q [生物科学]; N [自然科学总论];
学科分类号
07 ; 0710 ; 09 ;
摘要
Visual quality and algorithm efficiency are two main interests in video frame interpolation. We propose a hybrid task-based convolutional neural network for fast and accurate frame interpolation of 4K videos. The proposed method synthesizes low-resolution frames, then reconstructs high-resolution frames in a coarse-to-fine fashion. We also propose edge loss, to preserve high-frequency information and make the synthesized frames look sharper. Experimental results show that the proposed method achieves state-of-the-art performance and performs 2.69x faster than the existing methods that are operable for 4K videos, while maintaining comparable visual and quantitative quality.
引用
收藏
页数:15
相关论文
共 38 条
[1]  
[Anonymous], P 31 AAAI C ART INT
[2]  
Baker Simon, 2007, 2007 11th IEEE International Conference on Computer Vision, P1
[3]   High accuracy optical flow estimation based on a theory for warping [J].
Brox, T ;
Bruhn, A ;
Papenberg, N ;
Weickert, J .
COMPUTER VISION - ECCV 2004, PT 4, 2004, 2034 :25-36
[4]   Real-Time Video Super-Resolution with Spatio-Temporal Networks and Motion Compensation [J].
Caballero, Jose ;
Ledig, Christian ;
Aitken, Andrew ;
Acosta, Alejandro ;
Totz, Johannes ;
Wang, Zehan ;
Shi, Wenzhe .
30TH IEEE CONFERENCE ON COMPUTER VISION AND PATTERN RECOGNITION (CVPR 2017), 2017, :2848-2857
[6]  
Cheng H, 2017, IEEE INT C INT ROBOT, P3446, DOI 10.1109/IROS.2017.8206184
[7]  
Clevert D.-A., 2016, 4 INT C LEARN REPR I
[8]  
Deng J, 2009, PROC CVPR IEEE, P248, DOI 10.1109/CVPRW.2009.5206848
[9]   Learning a Deep Convolutional Network for Image Super-Resolution [J].
Dong, Chao ;
Loy, Chen Change ;
He, Kaiming ;
Tang, Xiaoou .
COMPUTER VISION - ECCV 2014, PT IV, 2014, 8692 :184-199
[10]   FlowNet: Learning Optical Flow with Convolutional Networks [J].
Dosovitskiy, Alexey ;
Fischer, Philipp ;
Ilg, Eddy ;
Haeusser, Philip ;
Hazirbas, Caner ;
Golkov, Vladimir ;
van der Smagt, Patrick ;
Cremers, Daniel ;
Brox, Thomas .
2015 IEEE INTERNATIONAL CONFERENCE ON COMPUTER VISION (ICCV), 2015, :2758-2766