Video Enhancement with Task-Oriented Flow

被引：4
作者：
Tianfan Xue
Baian Chen
Jiajun Wu
Donglai Wei
William T. Freeman
机构：
[1] Google Research,
[2] Massachusetts Institute of Technology,undefined
[3] Harvard University,undefined
[4] Google Research,undefined
来源：
International Journal of Computer Vision | 2019年 / 127卷
关键词：
Video processing; Optical flow; Neural network; Video dataset;
D O I：
暂无
中图分类号：
学科分类号：
摘要：
Many video enhancement algorithms rely on optical flow to register frames in a video sequence. Precise flow estimation is however intractable; and optical flow itself is often a sub-optimal representation for particular video processing tasks. In this paper, we propose task-oriented flow (TOFlow), a motion representation learned in a self-supervised, task-specific manner. We design a neural network with a trainable motion estimation component and a video processing component, and train them jointly to learn the task-oriented flow. For evaluation, we build Vimeo-90K, a large-scale, high-quality video dataset for low-level video processing. TOFlow outperforms traditional optical flow on standard benchmarks as well as our Vimeo-90K dataset in three video processing tasks: frame interpolation, video denoising/deblocking, and video super-resolution.
引用
收藏
页码：1106 / 1125
页数：19
相关论文
共 44 条
  • [1] Baker S(2011)A database and evaluation methodology for optical flow International Journal of Computer Vision 92 1-31
  • [2] Scharstein D(2010)Nonlocal video denoising, simplification and inpainting using discrete regularization on graphs Signal Process 90 2445-2455
  • [3] Lewis J(1981)Determining optical flow Artif Intell 17 185-203
  • [4] Roth S(2016)Video super-resolution with convolutional neural networks IEEE Transactions on Computational Imaging 2 109-122
  • [5] Black MJ(2014)On bayesian adaptive video super resolution IEEE Transactions on Pattern Analysis and Machine intelligence 36 346-360
  • [6] Szeliski R(2012)Video denoising, deblocking, and enhancement through separable 4-d nonlocal spatiotemporal transforms IEEE Transactions on Image Processing 21 3952-3966
  • [7] Ghoniem M(1998)Dense estimation and object-based segmentation of the optical flow with robust techniques IEEE Transactions on Image Processing 7 703-719
  • [8] Chahir Y(2014)Super-resolution: A comprehensive survey Machine Vision and Applications 25 1423-1468
  • [9] Elmoataz A(2001)Modeling the shape of the scene: A holistic representation of the spatial envelope International Journal of Computer Vision 42 145-175
  • [10] Horn BK(2010)Video denoising based on a spatiotemporal gaussian scale mixture model IEEE Transactions on Circuits and Systems for Video Technology 20 1032-1040
12345