Dual-stream VO: Visual Odometry Based on LSTM Dual-Stream Convolutional Neural Network

This paper studies the problem of visual odometry based on a deep recurrent convolutional neural network. A new visual odometry algorithm based on dual-stream convolutional neural networks with long short-term memory is proposed. The color stream of the convolutional neural network acquires the color features in the RGB image. The depth stream acquires the contour features in the depth image, generates fusion features through the feature fusion unit, and finally predicts the pose at the current moment through autonomous sequential modeling using recurrent neural networks. Experimental validation on the TUM dataset showed that the method introduces contour features into the system through a dual-stream architecture of neural networks, which provides higher accuracy and robust-ness compared to other convolutional neural network-based visual odometry systems, especially in the presence of motion blur and poor lighting.