Depth upsampling based on deep edge-aware learning

Depth map upsampling will unavoidably smoothen the edges leading to blurry results on the depth boundaries, especially at large upscaling factors. Given that edges represent the most important cue in addressing the task of depth upsampling, we propose a novel depth upsampling framework based on deep edge-aware learning. Unlike existing CNN-based approaches that directly predict depth values from low resolution (LR) depth input, our framework firstly learns edge information of depth boundaries from the known LR depth map and its corresponding high resolution (HR) color image as reconstruction cues. Then, two depth restoration modules, i.e., a fast depth filling strategy and a cascaded restoration network, are proposed to recover HR depth map by leveraging the predicted edge map and the HR color image. Extensive comparisons on both edge inference and depth upsampling under noisy and noiseless cases demonstrate the superiority of the proposed approaches. (C) 2020 Elsevier Ltd. All rights reserved.