DeepLiDAR: Deep Surface Normal Guided Depth Prediction for Outdoor Scene from Sparse LiDAR Data and Single Color Image

被引:253
作者
Qiu, Jiaxiong [1 ]
Cui, Zhaopeng [2 ]
Zhang, Yinda [3 ]
Zhang, Xingdi [1 ]
Liu, Shuaicheng [1 ,4 ]
Zeng, Bing [1 ]
Pollefeys, Marc [2 ,5 ]
机构
[1] UESTC, Chengdu, Sichuan, Peoples R China
[2] Swiss Fed Inst Technol, Zurich, Switzerland
[3] Google, Mountain View, CA 94043 USA
[4] Megvii Technol, Beijing, Peoples R China
[5] Microsoft, Albuquerque, NM USA
来源
2019 IEEE/CVF CONFERENCE ON COMPUTER VISION AND PATTERN RECOGNITION (CVPR 2019) | 2019年
关键词
SHAPE;
D O I
10.1109/CVPR.2019.00343
中图分类号
TP18 [人工智能理论];
学科分类号
081104 ; 0812 ; 0835 ; 1405 ;
摘要
In this paper, we propose a deep learning architecture that produces accurate dense depth for the outdoor scene from a single color image and a sparse depth. Inspired by the indoor depth completion, our network estimates surface normals as the intermediate representation to produce dense depth, and can be trained end-to-end. With a modified encoder-decoder structure, our network effectively fuses the dense color image and the sparse LiDAR depth. To address outdoor specific challenges, our network predicts a confidence mask to handle mixed LiDAR signals near foreground boundaries due to occlusion, and combines estimates from the color image and surface normals with learned attention maps to improve the depth accuracy especially for distant areas. Extensive experiments demonstrate that our model improves upon the state-of-the-art performance on KITTI depth completion benchmark. Ablation study shows the positive impact of each model components to the final performance, and comprehensive analysis shows that our model generalizes well to the input with higher sparsity or from indoor scenes.
引用
收藏
页码:3308 / 3317
页数:10
相关论文
共 58 条
[1]  
[Anonymous], P INT C 3D VIS 3DV
[2]  
[Anonymous], P EUR C COMP VIS ECC
[3]  
[Anonymous], 2018, P BRIT MACH VIS C BM
[4]  
[Anonymous], 2018, P IEEE INT C ROB AUT
[5]  
[Anonymous], IET COMPUTER VISION
[6]  
[Anonymous], 2015, P IEEE C COMP VIS PA
[7]  
[Anonymous], ELECT LETT
[8]  
[Anonymous], 2017, MULTIMEDIA TOOLS APP
[9]  
[Anonymous], 2014, CORR
[10]  
[Anonymous], 2012, 2012 IEEE COMP SOC C