Boosting Sparse Point Cloud Object Detection via Image Fusion

A point cloud is one of the primary sensing data in autonomous driving. Most 3D object detection algorithms rely heavily on a dense point cloud. Driven by the need for point cloud density, people have devoted considerable resources to pursuing high-resolution lidar systems and point cloud aggregation techniques. In this paper, we take a different view. We explore a new method to boost object detection on a relatively inexpensive sparse point cloud. Our method projects each lidar point on images and uses images to learn the point's local coordinates on the object. It then registers the predicted local coordinates with lidar coordinates to locate the object. At a minimum, our method can detect an object with a single lidar point on the object. Our experiments on the large-scale nuScenes dataset show we can significantly boost detection accuracy on sparse point cloud regions and improve detection distance.