Planar Array Lidar and Camera Calibration Method Based on Tetrahedral Features

: Compared with traditional mechanical multi-line Lidar, area array Lidar can achieve greater field of view coverage through non-repetitive scanning, and has many applications in industrial production and robotics. A single sensor often has some limitations, while the fusion of multi-sensor data has higher precision and accuracy. Point cloud data contains accurate depth information of the environment, while image data contains rich color and texture information of the environment. The fusion of Point cloud and image can increase the dimensionality of the sensor's perception of environmental, and also enable robots and automation equipment to achieve more complex tasks and reconstruct colorful three-dimensional environments. The prerequisite for achieving the fusion of point cloud data and image data is to calibrate the extrinsic parameters of the Lidar and camera. The purpose of sensors external parameter calibration is to find the accurate position conversion relationship between the two sensor coordinate systems. In order to improve the calibration accuracy of array Lidar and monocular camera, this paper proposes a calibration method based on tetrahedral structure for array Lidar and camera. The tetrahedron structure formed by two isosceles right triangle calibration plates and the ground is used as the calibration object. Using the random sampling consensus algorithm to extract three plane features from point cloud data, The straight line parameter equations of the edges of the tetrahedral structure are obtained through the intersection of the three planes. The common points of the three planes are the vertices of the tetrahedral structure. The line segment detection algorithm is used to extract the straight line features of the intersection of three planes in the image data, and the vertex coordinates of the tetrahedral structure are obtained through the straight line intersection. The method of indirectly obtaining point and linear features through plane extraction is more accurate than the direct extraction method. The back-projection rays of the corner points in the image and the straight-line features in the point cloud form multiple sets of skew lines. The distance between the skew lines is used to construct the residual equation of the rotation and translation relationship. The angle error between the projection of the straight line feature in the point cloud on the image and the real imaging straight line is used to establish the residual equation of the rotation relationship. The "point-three-line" correspondence relationship between point cloud data and image data is established, which improves the constraint strength between corresponding features in point cloud and image data. Based on the Rodriguez formula, a rotation vector is used to represent the transformation matrix between the Lidar and camera coordinate systems. The transformation vector between the camera and radar coordinate systems is solved through nonlinear optimization and the transformation matrix is finally obtained. The distance from the projected point of the edge in the point cloud to the edge in the image is used as the projection error from the point cloud data to the image data. The proposed method is experimentally compared with the Livox company' s open-source calibration method and the University of Hong Kong's targetless calibration method. The tetrahedral structure data collected in this paper and the open-source algorithm data were tested, respectively. The calibration results on images with a resolution of 1 920x1 080 shows that the average projection error of the proposed method is within 0.6 pixels, which is superior to the other two calibration methods. At the same time, without a specific tetrahedral structure calibration object. The proposed method can use the wall corner tetrahedral structure to complete the calibration, which reflects the flexibility of the proposed method. The proposed high-precision calibration method of Lidar and camera provides a research foundation for the subsequent research on the fusion of Lidar data and image data.