UnOS: Unified Unsupervised Optical-flow and Stereo-depth Estimation by Watching Videos

In this paper, we propose UnOS, an unified system for unsupervisedopticalflow and stereo depth estimation using convolutional neural network (CNN) by taking advantages of theirinherentgeometricalconsistency based on the rigidscene assumption [31]. UnOS significantly outperforms other state-of-the-art(SOTA) unsupervisedapproachesthat treatedthe two tasks independently. Specifically, given two consecutive stereo image pairsfrom a video, UnOS estimates per-pixel stereo depth images, camera ego-motion and opticalflow with three parallelCNNs. Based on these quantities, UnOS computes rigid optical flow and compares it againstthe opticalflow estimatedfrom the FlowNet, yielding pixels satisfying the rigid-sceneassumption. Then, we encourage geometricalconsistency between the two estimated flows within rigid regions, from which we derive a rigid-aware direct visual odometry (RDVO) module. We also propose rigid and occlusion-awareflow-consistency losses for the learning of UnOS. We evaluated our results on the popularKITTI datasetover 4 related tasks, i.e. stereo depth, opticalflow, visual odometry and motion segmentation.