Scene Flow Estimation Through 3D Analysis of Multi-Focus Images

If scene flow expressed in three-dimensional (3D) vector fields is robustly estimated from multi-view or multi-focus images, we can develop advanced 3D motion tracking and motion compensation for 3D video compression. In this study, based on a synthesis of multi-focus images from multi-view images, we propose a novel method for analyzing 3D scene flow accurately at low computational cost as an extension of 2D optical flow estimation. Our method directly estimates 3D scene flow in multi focus images arranged in a direction orthogonal to them by an extended pyramidal Lucas-Kanade method for 3D. Experimental results show that scene flow estimation is achieved efficiently when using our proposed method. In addition, our method can be easily implemented on a graphics processing unit for real-time applications.