Visual-LiDAR-Inertial Odometry: A New Visual-Inertial SLAM Method based on an iPhone 12 Pro

As today's smartphone integrates various imaging sensors and Inertial Measurement Units (IMU) and becomes computationally powerful, there is a growing interest in developing smartphone-based visual-inertial (VI) SLAM methods for robotics and computer vision applications. In this paper, we introduce a new SLAM method, called Visual-LiDAR-Inertial Odometry (VLIO), based on an iPhone 12 Pro. VLIO formulates device pose estimation as an optimization problem that minimizes a cost function based on the residuals of the inertial, visual, and depth measurements. We present the first work that 1) characterizes the iPhone's LiDAR in depth measurement and identifies the models for the measurement error and standard deviation, and 2) characterizes pose change estimation with LiDAR data. The measurement models are then used to compute the depth-related and visual-feature-related residuals for the cost function. Also, VLIO tracks varying camera intrinsic parameters (CIP) in real-time and uses them in computing these residuals. Both approaches result in more accurate residual terms and thus more accurate pose estimation. The CIP tracking method eliminates the need of a sophisticated model-fitting process that includes camera calibration and paring of the CIPs and IMU measurements with various phone orientations. Experimental results validate the efficacy of VLIO.