Deep Learning-based 3D Object Detection Using LiDAR and Image Data Fusion

Robust environmental perception systems are key to implementing advanced driver assistance features in modern vehicles. To acquire knowledge of the environment, vehicles employ sensors like LiDAR, Camera and Radar for object detection and tracking. For autonomous driving applications (SAE J3016 Level 3 and above), LiDAR and camera-based systems are most common. Camera-based systems have difficulty in operating under low-visibility conditions while LiDAR-based systems do not perform well in detecting objects at farther distances due to sparse point clouds. This work presents an approach to improving the shortcomings of the LiDAR-based detection systems by upsampling sparse LiDAR point clouds and then fusing the camera data to it to improve the quality of the point clouds generated that can be then used for object detection and identification. The proposed technique has been validated on the KITTI vision benchmark dataset. Results indicate that the proposed system is able to outperform standalone LiDAR-based 3D Object detection.