A non-parametric depth modification model for registration between color and depth images

Despite its most popularity among all depth cameras in the computer vision applications, the Microsoft Kinect sensor suffers from low depth accuracy. In this work we propose a novel non-parametric depth modification model to improve the depth accuracy of the Kinect sensor by iteratively registering depth images and color images. In particular, we first establish a coarse correspondence based on the feature descriptor of the canny edge at each iteration, and estimate the fine correspondence using an L2E\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$L_2E$$\end{document} algorithm. We utilize the non-parametric Gaussian mixture model to replace the Gaussian single model and build the regularization term to constrain the correlations between functions. Then, based on the correspondence results, the depth data are corrected and optimized. Extensive experiments have been performed to verify the effectiveness of the proposed approach, and the results have demonstrated that our method is able to greatly enhance the depth accuracy of the Kinect sensor compared with baseline methods.