Two-Step LiDAR/Camera/IMU Spatial and Temporal Calibration Based on Continuous-Time Trajectory Estimation

Accurate intersensor spatiotemporal transformation is the fundamental prerequisite for multisensor fusion. However, traditional discrete-time calibration methods have natural shortcomings when dealing with distorted and asynchronous raw measurements collected by light detection and ranging (LiDAR) and rolling shutter cameras. To this end, we propose a two-step LiDAR/camera/inertial measurement unit (IMU) spatiotemporal calibration method using continuous-time batch optimization. This proposed method is target free and has no assumptions about the sensor configuration. Specifically, the LiDAR point-to-plane, gyroscope, and accelerometer factors are jointly optimized to obtain the optimal LiDAR-IMU extrinsic parameters, time offset, and poses of control points. Subsequently, combined with visual reprojection factor, the control points in the spline trajectory will be fixed to obtain the camera-IMU extrinsic parameters as well as its time offset. A series of experiments in both simulated and real-world environments were conducted to evaluate the effectiveness of the proposed calibration method. Moreover, the effects of different numbers of iterations, environments (indoor and outdoor), IMU sampling rates, and control point frequencies on the calibration performance are analyzed in detail.