SSG-VINS: A Semantic Segmentation-Guided Adaptive Visual-Inertial State Estimator

Despite extensive research in visual SLAM leveraging semantic segmentation techniques, many approaches primarily use semantic labels to handle only dynamic objects. Some recent methods have incorporated semantic segmentation into the SLAM system by quantifying semantic class differences. However, scoring differences is not straightforward and computationally expensive. Additionally, photometric noise caused by illumination changes and varying camera viewpoints can lead to false positives in feature matching and loop closing. To overcome these challenges, we present a robust visual-inertial state estimator named SSG-VINS, which integrates semantic segmentation throughout the algorithm to improve robustness and accuracy. Key contributions include efficient utilization of semantic segmentation for state estimation, a novel filter for reliable feature tracking, robust weighted optimization emphasizing long-persistent features, and a filter to reduce false positives in loop detection. We assess the accuracy of our framework using the uHuman2 dataset and compare its performance to another state-of-the-art algorithm. Results demonstrate that SSG-VINS consistently outperforms the leading method, providing more precise state estimation and reducing false positives in various environments.