GRPoseNet: a generalizable and robust 6D object pose estimation network using sparse RGB views

Six-degree-of-freedom object pose estimation plays a crucial role in various computer vision and robotics tasks. Existing methods often rely heavily on CAD models and substantial prior information, limiting their generalization to unseen objects in open scenes. To address this limitation, we propose GRPoseNet, a generalizable and robust 6D object pose estimation network that can predict the pose of unseen objects using only sparse RGB images with reference poses. GRPoseNet comprises an open-world detector, a viewpoint selector, and an adaptive multi-scale refiner. The open-world detector leverages pre-trained large models for zero-shot segmentation and feature extraction, overcoming detection and matching errors with unseen objects. The viewpoint selector uses our designed similarity network to select the most similar reference view for initial pose estimation. The adaptive multi-scale refiner further refines the pose by iteratively updating rotation and translation residuals based on multi-scale features and adaptive weights. Extensive experiments on benchmark datasets and our robust test dataset, RBMOP, demonstrate that GRPoseNet achieves state-of-the-art performance, showing excellent generalization and robustness to unseen objects and sparse views. The codes and datasets are available at: https://github.com/KierSaS/GRPoseNet.