3D object detection based on fusion of image and point cloud in autonomous driving traffic scenarios

In order to improve the accuracy of 3D object detection in autonomous driving traffic Scenarios, this paper proposes a 3D object detection method that integrates feature pyramid structure FPN (Feature Pyramid Network) and frustum attention module by fusing image and point cloud data. Firstly, the 2D object detection result of the image is projected into the point cloud and the redundant point cloud is trimmed to generate the 3D data of the frustum with the semantic information of the image; Secondly, according to the distribution pattern of point cloud in the frustum, linearly adjust and generate the sliding stride and height of the frustum sequence; Then, in order to improve the detection accuracy of targets at different scales, a multi-scale 3D object detection module was constructed based on the feature pyramid structure FPN and the fully convolutional network (FCN) to improve the feature extraction ability of the detection model; Next, to suppress the impact of invalid frustum sequences on detection accuracy, it is proposed to incorporate frustum attention modules into the detection model; Finally, experiments were conducted on the KITTI, and the results showed that the proposed improved model improved vehicle detection accuracy by 0.88%, 1.53%, and 2.33%, pedestrian detection accuracy by 0.99%, 1.88%, and 0.10%, and cyclist detection accuracy by 1.18%, 3.08%, and 2.78%, respectively, under the three occlusion types of easy, medium, and difficult occlusion, effectively improving the 3D object detection accuracy in autonomous driving traffic scenes. © The Author(s), under exclusive licence to Springer Science+Business Media, LLC, part of Springer Nature 2024.