HI-SLAM: Monocular Real-Time Dense Mapping With Hybrid Implicit Fields

被引:12
作者
Zhang, Wei [1 ,2 ]
Sun, Tiecheng [3 ]
Wang, Sen [2 ,4 ]
Cheng, Qing [2 ,4 ]
Haala, Norbert [1 ]
机构
[1] Univ Stuttgart, Inst Photogrammetry, D-70174 Stuttgart, Germany
[2] Huawei Munich Res Ctr, D-80992 Munich, Germany
[3] Cent Media Technol Inst, Huawei Labs 2012, Chengdu 611731, Peoples R China
[4] Tech Univ Munich, D-80333 Munich, Germany
关键词
Deep learning for visual perception; mapping; SLAM;
D O I
10.1109/LRA.2023.3347131
中图分类号
TP24 [机器人技术];
学科分类号
080202 ; 1405 ;
摘要
In this letter, we present a neural field-based real-time monocular mapping framework for accurate and dense Simultaneous Localization and Mapping (SLAM). Recent neural mapping frameworks show promising results, but rely on RGB-D or pose inputs, or cannot run in real-time. To address these limitations, our approach integrates dense-SLAM with neural implicit fields. Specifically, our dense SLAM approach runs parallel tracking and global optimization, while a neural field-based map is constructed incrementally based on the latest SLAM estimates. For the efficient construction of neural fields, we employ multi-resolution grid encoding and signed distance function (SDF) representation. This allows us to keep the map always up-to-date and adapt instantly to global updates via loop closing. For global consistency, we propose an efficient Sim(3)-based pose graph bundle adjustment (PGBA) approach to run online loop closing and mitigate the pose and scale drift. To enhance depth accuracy further, we incorporate learned monocular depth priors. We propose a novel joint depth and scale adjustment (JDSA) module to solve the scale ambiguity inherent in depth priors. Extensive evaluations across synthetic and real-world datasets validate that our approach outperforms existing methods in accuracy and map completeness while preserving real-time performance.
引用
收藏
页码:1548 / 1555
页数:8
相关论文
共 37 条
[1]   Neural RGB-D Surface Reconstruction [J].
Azinovic, Dejan ;
Martin-Brualla, Ricardo ;
Goldman, Dan B. ;
Niessner, Matthias ;
Thies, Justus .
2022 IEEE/CVF CONFERENCE ON COMPUTER VISION AND PATTERN RECOGNITION (CVPR 2022), 2022, :6280-6291
[2]   ORB-SLAM3: An Accurate Open-Source Library for Visual, Visual-Inertial, and Multimap SLAM [J].
Campos, Carlos ;
Elvira, Richard ;
Gomez Rodriguez, Juan J. ;
Montiel, Jose M. M. ;
Tardos, Juan D. .
IEEE TRANSACTIONS ON ROBOTICS, 2021, 37 (06) :1874-1890
[3]  
Choi S, 2015, PROC CVPR IEEE, P5556, DOI 10.1109/CVPR.2015.7299195
[4]   Orbeez-SLAM: A Real-time Monocular Visual SLAM with ORB Features and NeRF-realized Mapping [J].
Chung, Chi-Ming ;
Tseng, Yang-Che ;
Hsu, Ya-Ching ;
Shi, Xiang-Qian ;
Hua, Yun-Hung ;
Yeh, Jia-Fong ;
Chen, Wen-Chin ;
Chen, Yi-Ting ;
Hsu, Winston H. .
2023 IEEE INTERNATIONAL CONFERENCE ON ROBOTICS AND AUTOMATION (ICRA 2023), 2023, :9400-9406
[5]   ScanNet: Richly-annotated 3D Reconstructions of Indoor Scenes [J].
Dai, Angela ;
Chang, Angel X. ;
Savva, Manolis ;
Halber, Maciej ;
Funkhouser, Thomas ;
Niessner, Matthias .
30TH IEEE CONFERENCE ON COMPUTER VISION AND PATTERN RECOGNITION (CVPR 2017), 2017, :2432-2443
[6]   Fast Monocular Scene Reconstruction with Global-Sparse Local-Dense Grids [J].
Deng, Wei ;
Choy, Chris ;
Loop, Charles ;
Litany, Or ;
Zhu, Yuke ;
Anandkurnar, Anima .
2023 IEEE/CVF CONFERENCE ON COMPUTER VISION AND PATTERN RECOGNITION, CVPR, 2023, :4263-4272
[7]   Omnidata: A Scalable Pipeline for Making Multi-Task Mid-Level Vision Datasets from 3D Scans [J].
Eftekhar, Ainaz ;
Sax, Alexander ;
Malik, Jitendra ;
Zamir, Amir .
2021 IEEE/CVF INTERNATIONAL CONFERENCE ON COMPUTER VISION (ICCV 2021), 2021, :10766-10776
[8]   LSD-SLAM: Large-Scale Direct Monocular SLAM [J].
Engel, Jakob ;
Schoeps, Thomas ;
Cremers, Daniel .
COMPUTER VISION - ECCV 2014, PT II, 2014, 8690 :834-849
[9]   Neural 3D Scene Reconstruction with the Manhattan-world Assumption [J].
Guo, Haoyu ;
Peng, Sida ;
Lin, Haotong ;
Wang, Qianqian ;
Zhang, Guofeng ;
Bao, Hujun ;
Zhou, Xiaowei .
2022 IEEE/CVF CONFERENCE ON COMPUTER VISION AND PATTERN RECOGNITION (CVPR 2022), 2022, :5501-5510
[10]   ESLAM: Efficient Dense SLAM System Based on Hybrid Representation of Signed Distance Fields [J].
Johari, Mohammad Mahdi ;
Carta, Camilla ;
Fleuret, Francois .
2023 IEEE/CVF CONFERENCE ON COMPUTER VISION AND PATTERN RECOGNITION (CVPR), 2023, :17408-17419