Efficient Dynamic Scene Deblurring Using Spatially Variant Deconvolution Network with Optical Flow Guided Training

被引:94
作者
Yuan, Yuan
Su, Wei
Ma, Dandan [1 ]
机构
[1] Northwestern Polytech Univ, Sch Comp Sci, Xian, Shaanxi, Peoples R China
来源
2020 IEEE/CVF CONFERENCE ON COMPUTER VISION AND PATTERN RECOGNITION (CVPR) | 2020年
基金
中国国家自然科学基金;
关键词
D O I
10.1109/CVPR42600.2020.00361
中图分类号
TP18 [人工智能理论];
学科分类号
081104 ; 0812 ; 0835 ; 1405 ;
摘要
In order to remove the non-uniform blur of images captured from dynamic scenes, many deep learning based methods design deep networks for large receptive fields and strong fitting capabilities, or use multi-scale strategy to deblur image on different scales gradually. Restricted by the fixed structures and parameters, these methods are always huge in model size to handle complex blurs. In this paper, we start from the deblurring deconvolution operation, then design an effective and real-time deblurring network. The main contributions are three folded, 1) we construct a spatially variant deconvolution network using modulated deformable convolutions, which can adjust receptive fields adaptively according to the blur features. 2) our analysis shows the sampling points of deformable convolution can be used to approximate the blur kernel, which can be simplified to bi-directional optical flows. So the position learning of sampling points can be supervised by bi-directional optical flows. 3) we build a light-weighted backbone for image restoration problem, which can balance the calculations and effectiveness well. Experimental results show that the proposed method achieves state-of-the-art deblurring performance, but with less parameters and shorter running time.
引用
收藏
页码:3552 / 3561
页数:10
相关论文
共 37 条
[1]  
[Anonymous], IEEE I CONF COMP VIS, DOI DOI 10.1109/ICCV.2015.123
[2]  
[Anonymous], 2015, ACS SYM SER
[3]  
Chen HJ, 2018, IEEE INT CONF COMPUT
[4]   Deformable Convolutional Networks [J].
Dai, Jifeng ;
Qi, Haozhi ;
Xiong, Yuwen ;
Li, Yi ;
Zhang, Guodong ;
Hu, Han ;
Wei, Yichen .
2017 IEEE INTERNATIONAL CONFERENCE ON COMPUTER VISION (ICCV), 2017, :764-773
[5]   FlowNet: Learning Optical Flow with Convolutional Networks [J].
Dosovitskiy, Alexey ;
Fischer, Philipp ;
Ilg, Eddy ;
Haeusser, Philip ;
Hazirbas, Caner ;
Golkov, Vladimir ;
van der Smagt, Patrick ;
Cremers, Daniel ;
Brox, Thomas .
2015 IEEE INTERNATIONAL CONFERENCE ON COMPUTER VISION (ICCV), 2015, :2758-2766
[6]   Two-frame motion estimation based on polynomial expansion [J].
Farnebäck, G .
IMAGE ANALYSIS, PROCEEDINGS, 2003, 2749 :363-370
[7]  
Glorot X., 2010, P INT C ART INT STAT, P249
[8]   From Motion Blur to Motion Flow: a Deep Learning Solution for Removing Heterogeneous Motion Blur [J].
Gong, Dong ;
Yang, Jie ;
Liu, Lingqiao ;
Zhang, Yanning ;
Reid, Ian ;
Shen, Chunhua ;
van den Hengel, Anton ;
Shi, Qinfeng .
30TH IEEE CONFERENCE ON COMPUTER VISION AND PATTERN RECOGNITION (CVPR 2017), 2017, :3806-3815
[9]  
Goodfellow IJ, 2014, ADV NEURAL INFORM PR, V27, P2672, DOI DOI 10.1145/3422622
[10]   FlowNet 2.0: Evolution of Optical Flow Estimation with Deep Networks [J].
Ilg, Eddy ;
Mayer, Nikolaus ;
Saikia, Tonmoy ;
Keuper, Margret ;
Dosovitskiy, Alexey ;
Brox, Thomas .
30TH IEEE CONFERENCE ON COMPUTER VISION AND PATTERN RECOGNITION (CVPR 2017), 2017, :1647-1655