Camera-LiDAR Fusion for Object Detection，Tracking and Prediction

Objectives: A real-time and robust 3D dynamic object perception module is a key part of autono⁃ mous driving system. Methods: This paper fuses monocular camera and light detection and ranging (LiDAR) to detect 3D objects. First, we use convolutional neural network to detect 2D bounding boxes and generate 3D frustum region of interest (ROI) according to the geometric projection relation between camera and LiDAR. Then, we cluster the point cloud in the frustum ROI and fit the 3D bounding box of the objects. After detecting 3D objects, we reidentify the objects between adjacent frames by appearance features and Hungarian algorithm, and then propose a tracker management model based on a quad-state machine. Finally, a novel prediction model is proposed, which leverages lane lines to constrain vehicle trajectories. Results: The experimental results show that in the stage of target detection, the accuracy and recall of the proposed algorithm can reach 92.5% and 86.7%, respectively. The root mean square error of the proposed trajectory prediction algorithm is smaller than that of the existing algorithms on the simulation datasets in⁃ cluding straight line, arc and spiral curves. The whole algorithm only takes approximately 25 ms, which meets the real-time requirements. Conclusions: The proposed algorithm is effective and efficient, and has a good performance in different lane lines. © 2024 Editorial Department of Geomatics and Information Science of Wuhan University. All rights reserved.