RGB-D SLAM in Dynamic Environments with Multilevel Semantic Mapping

Dynamic environments pose a severe challenge to visual SLAM as moving objects invalidate the assumption of a static background. While recent works employ deep learning to address the challenge, they still fail to determine whether an object actually moves or not, resulting in the misguidance of object tracking and background reconstruction. Hence we design a SLAM system to simultaneously estimate trajectory and construct object-level dense 3D semantic maps in dynamic environments. Synergizing deep learning-based object detection, we leverage geometric constraints by using optical flow and the relationship between objects to identify those moving but predefined static objects. To construct more precise 3D semantic maps, our method employs an unsupervised algorithm to segment 3D point cloud generated by depth data into meaningful clusters. The 3D point clusters are then synergized with semantic cues generated by deep learning to produce a more accurate 3D semantic map. We evaluate the proposed system on TUM RGB-D dataset and ICL-NUIM dataset as well as in real-world indoor environments. Qualitative and quantitative experiments show that our method outperforms state-of-the-art approaches in various dynamic scenes in terms of both accuracy and robustness.