MonoScene: Monocular 3D Semantic Scene Completion

被引:138
作者
Anh-Quan Cao [1 ]
de Charette, Raoul [1 ]
机构
[1] INRIA, Paris, France
来源
2022 IEEE/CVF CONFERENCE ON COMPUTER VISION AND PATTERN RECOGNITION (CVPR 2022) | 2022年
关键词
D O I
10.1109/CVPR52688.2022.00396
中图分类号
TP18 [人工智能理论];
学科分类号
081104 ; 0812 ; 0835 ; 1405 ;
摘要
MonoScene proposes a 3D Semantic Scene Completion (SSC) framework, where the dense geometry and semantics of a scene are inferred from a single monocular RGB image. Different from the SSC literature, relying on 2.5 or 3D input, we solve the complex problem of 2D to 3D scene reconstruction while jointly inferring its semantics. Our framework relies on successive 2D and 3D UNets, bridged by a novel 2D-3D features projection inspired by optics, and introduces a 3D context relation prior to enforce spatio-semantic consistency. Along with architectural contributions, we introduce novel global scene and local frustums losses. Experiments show we outperform the literature on all metries and datasets while hallucinating plausible scenery even beyond the camera field of view. Our code and trained models are available at https://github.com/cv-rits/MonoScene.
引用
收藏
页码:3981 / 3991
页数:11
相关论文
共 83 条
[1]  
[Anonymous], 2020, CVPR, DOI DOI 10.1109/CVPR42600.2020.00093
[2]  
[Anonymous], 2019, CVPR, DOI DOI 10.1109/CVPR.2019.01112
[3]  
[Anonymous], 2018, NeurIPS
[4]  
[Anonymous], 1963, PhD Thesis
[5]  
[Anonymous], 2019, ICML
[6]  
[Anonymous], 2018, ICML
[7]  
[Anonymous], 2020, CVPR, DOI DOI 10.1109/CVPR42600.2020.00013
[8]  
Avetisyan Armen, 2019, CVPR
[9]  
Behley J., 2019, ICCV
[10]  
Bhat Shariq Farooq, 2021, CVPR