Research of the three-dimensional tracking and registration method based on multiobjective constraints in an AR system

To match the virtual image and actual environment in an augmented reality (AR) system, it is necessary to complete the task of three-dimensional (3D) tracking registration. This paper proposes a new method for 3D tracking registration. Previous methods extract feature points in images to realize tracking registration. In this paper, the objects are extracted from the deep convolution neural network in the scene, and the camera pose is estimated by establishing the constraint relation of the objects. Then, 3D tracking and registration of the virtual object are realized. We design an improved single-shot multibox detector semantic segmentation network to identify and segment the scene and extract the pixel classification results of the objects in the scene. The effect of classification with this method is better. The depth of the extracted object is estimated based on the data from the left and right cameras, and the 2D image is converted into a 3D point cloud. A camera pose estimation method, combined with multiobjective information, is proposed. The camera transformation matrix is directly estimated by establishing a mathematical model. This method avoids the effect on the accuracy of the camera pose estimation when the feature points are not sufficient. Moreover, by assigning different weights to the point clouds of different objects, errors caused by the model can be reduced. The experimental results showed that the 3D registration method proposed in this paper is less than 2.5 pixels in the application scene of an augmented reality head-up display. This method had a better effect compared with that of existing methods and also improved driving safety. (C) 2018 Optical Society of America