Proving the efficiency of template matching-based markerless tracking methods which consider the camera perspective deformations

Template matching techniques are often used for camera tracking. They provide a good balance between computational cost and robustness to illumination changes. However, they lack robustness to camera orientation and scale changes. Camera movement, and specially rotation, generates perspective deformations that affect the process of patch matching, so the number of inliers (3D-2D correspondences) decreases. This fact affects camera tracking stability. This paper provides the following statistical proof: considering surface normals associated with 3D points substantially increases the number of inliers. So, this paper shows that computing perspective compensation improves the tracking. For instance, in a particular camera path used for experiments in this paper, without compensation, only a of 3D points projected into the image were found as inliers, while perspective compensation increased that figure up to a . These results must be contextualized in the analysis provided by the paper.