Investigating 3D object detection using stereo camera and LiDAR fusion with bird's-eye view representation

Multi-sensor fusion for 3D object detection is a crucial development in autonomous vehicle technology. Current research primarily explores the combination of monocular cameras with LiDARs. However, there is a notable gap in the integration of stereo cameras with LiDARs. A significant drawback of the fusion between monocular cameras and LiDARs is its vulnerability to sensor failures, which can completely disable the system. This paper presents a novel end-to-end framework called SLBEVFusion, which leverages the effective combination of stereo cameras and LiDARs for 3D object detection. The framework utilizes the rich semantic and depth data from stereo images along with spatial data from LiDAR point clouds to improve detection accuracy. Moreover, it addresses sensor failures by isolating the data streams of each sensor and implementing straightforward feature fusion in Bird's Eye View (BEV) space. Comprehensive testing on the challenging KITTI dataset demonstrates that our approach significantly enhances performance in 3D object detection by merging stereo cameras with LiDARs.