Online LiDAR-camera extrinsic parameters self-checking and recalibration

During production, smart cars are equipped with calibrated LiDARs and cameras. However, due to the vibrations that inevitably occur during driving, the sensors' extrinsic parameters may change slightly over time. It is a significant challenge to ensure the ongoing security of these systems throughout the car's lifetime. To address this issue, we propose a self-checking and recalibration algorithm that can continuously detect the sensor data of intelligent vehicles. If the sensor's miscalibration is detected, the data can be repaired promptly to ensure the vehicle's reliability. Our self-checking algorithm extracts features from the point cloud and image and performs pixel-wise comparisons. To improve feature quality, we utilize the patch-wise transformer to enhance local information exchange, which also benefits the subsequent extrinsic recalibration. To facilitate the study, we generate two customized datasets from the KITTI dataset and the Waymo Open Dataset. The experiments conducted on these datasets demonstrate the effectiveness of our proposed method in accurately calibrating the LiDAR and camera systems throughout the car's lifetime. This study is the first to highlight the importance of continually checking the calibrated extrinsic parameters for autonomous driving. Our findings contribute to the broader goal of improving safety and reliability in autonomous driving systems. The dataset and code are available at https://github.com/OpenCalib/LiDAR2camera_self-check.