3DTRIP: A General Framework for 3D Trajectory Recovery Integrated With Prediction

In an autonomous driving system, recovering the moving trajectories of the road users and predicting their future trend can greatly assist further driving decision-making. However, there are still many difficulties to solve this problem, due to the insufficient accuracy of the sensors and occlusion between objects. In this work, we propose a general framework for 3D trajectory recovery integrated with prediction (called 3DTRIP) based on deep learning methods through sensor fusion. The proposed method contains two modules: 1) trajectory recovery module to recover trajectories of moving objects from RGB and point cloud data and 2) trajectory prediction module to predict the future trajectories of objects. In the recovery module, we combine 2D multi-object tracker with 3D object detector to locate the accurate positions in trajectory, and propose a post-process method to further refine the results. In the prediction module, we apply Multilayer Perceptron (MLP) to extract global features of historical trajectories, and combine them with local features extracted by sequential model. Finally, we integrate these two modules to improve the accuracy of trajectory recovery. This framework helps autonomous cars to capture the historical movement of road users and predict their movement trends. The novelty lies in the efficient integration mechanism of trajectory recovery and prediction, making the trajectory recovery more accurate. We validate the method on KITTI tracking dataset and NuScenes dataset. The extensive experiments show improvements on trajectory recovery with the aid of prediction.