A real-time semi-dense depth-guided depth completion network

Depth completion, the task of predicting dense depth maps from given depth maps of sparse, is an important topic in computer vision. To cope with the task, both traditional image processing- based and data-driven deep learning-based algorithms have been established in the literature. In general, traditional algorithms, built upon non-learnable methods such as interpolation and custom kernels, can handle well flat regions but may blunt sharp edges. Deep learning-based algorithms, despite their strengths in many aspects, still have several limits, e.g., their performance depends heavily on the quality of the given sparse maps, and the dense maps they produced may contain artifacts and are often poor in terms of geometric consistency. To tackle these issues, in this work we propose a simple yet effective algorithm that aims to combine the strengths of both the traditional image processing techniques and the prevalent deep learning methods. Namely, given a sparse depth map, our algorithm first generates a semi-dense map and a 3D pose map using the adaptive densification module (ADM) and the coordinate projection module (CPM), respectively, and then input the obtained maps into a two-branch convolutional neural network so as to produce the final dense depth map. The proposed algorithm is evaluated on both challenging outdoor dataset: KITTI and indoor dataset: NYUv2, the experimental results show that our method performs better than some existing methods.