Research Progress of Deep Learning Based Optical Flow Computation Technology

Optical flow computation is an important research direction in image processing and computer vision. With the rapid development of the deep learning technology, the convolutional neural network based deep learning theories and methodologies have been the research focus of optical flow computation. This article mainly reviews the research progress of the deep learning based optical flow estimation technologies. First, the typical models and training strategies of the optical flow computing networks with supervised learning, unsupervised learning and semi-supervised learning are introduced. Second, the optimization methods of various network models are described and analyzed. Third, the evaluation benchmarks of Middlebury, MPI-Sintel and KITTI databases are summarized, and the experimental comparison results and analysis between the different deep-learning and variational optical flow methods are conducted. Finally, we discuss some issues of the deep learning based optical flow computation technology including the model complexity and generalization, the robustness of optical flow estimation and the accuracy of the small sample training. Afterwards, we point out several possible solutions and research ideas to address the above mentioned issues. © 2020, Chinese Institute of Electronics. All right reserved.