Robot Visual Servoing Grasping Based on Top-Down Keypoint Detection Network

被引:4
作者
Luo, Junqi [1 ,2 ]
Zhu, Liucun [2 ]
Li, Liang [2 ]
Hong, Peitao [2 ]
机构
[1] Guangxi Univ, Coll Mech Engn, Nanning 530004, Peoples R China
[2] Beibu Gulf Univ, Adv Sci & Technol Res Inst, Qinzhou 535000, Guangxi, Peoples R China
关键词
Domain randomization; keypoint detection network; robot grasping; visual servoing; KALMAN FILTER; VISION; ROBUST; SYSTEM;
D O I
10.1109/TIM.2023.3335521
中图分类号
TM [电工技术]; TN [电子技术、通信技术];
学科分类号
0808 ; 0809 ;
摘要
The paradigm of "deep-learning visual perception + hand-eye transformation + motion planning" for robot grasping has demonstrated viable capabilities in specific scenarios. However, its further development faces challenges in handling complex and dynamic environments. This article proposes a keypoint detection network-driven visual servoing grasping framework. First, we develop an efficient two-stage keypoint detector to perform real-time inference of sparse image-plane features for the target. Subsequently, a low-pass filtering algorithm is employed to smoothen the detected keypoints. These processed keypoints are then used in an image-based visual servoing (IBVS) controller to calculate the robot joint velocities, enabling precise tracking. A specialized dataset for training and evaluation was constructed using domain randomization techniques, comprising 11 K samples across six categories. Comprehensive experiments demonstrate the detector's low latency and accurate performance, even in low lighting, overexposure, partial occlusion, and densely packed environments. Static and dynamic grasping experiments validate that this framework achieves localization accuracy superior to five pixels and an overall grasping success rate exceeding 70% under unknown hand-eye calibration. The dataset is provided at github.com/hijunqi/VS_grasping_keypoint_detection_dataset.
引用
收藏
页码:1 / 11
页数:11
相关论文
共 34 条
[1]   Robust Multispectral Visual-Inertial Navigation With Visual Odometry Failure Recovery [J].
Beauvisage, Axel ;
Ahiska, Kenan ;
Aouf, Nabil .
IEEE TRANSACTIONS ON INTELLIGENT TRANSPORTATION SYSTEMS, 2022, 23 (07) :9089-9101
[2]   RAUM-VO: Rotational Adjusted Unsupervised Monocular Visual Odometry [J].
Cimarelli, Claudio ;
Bavle, Hriday ;
Sanchez-Lopez, Jose Luis ;
Voos, Holger .
SENSORS, 2022, 22 (07)
[3]   The Battle for Filter Supremacy: A Comparative Study of the Multi-State Constraint Kalman Filter and the Sliding Window Filter [J].
Clement, Lee E. ;
Peretroukhin, Valentin ;
Lambert, Jacob ;
Kelly, Jonathan .
2015 12TH CONFERENCE ON COMPUTER AND ROBOT VISION CRV 2015, 2015, :23-30
[4]   An improved SLAM based on RK-VIF: Vision and inertial information fusion via Runge-Kutta method [J].
Cui, Jia-shan ;
Zhang, Fang-rui ;
Feng, Dong-zhu ;
Li, Cong ;
Li, Fei ;
Tian, Qi-chen .
DEFENCE TECHNOLOGY, 2023, 21 :133-146
[5]   Adaptive Unscented Kalman Filter Based Estimation and Filtering for Dynamic Positioning with Model Uncertainties [J].
Deng, Fang ;
Yang, Hua-Lin ;
Wang, Long-Jin .
INTERNATIONAL JOURNAL OF CONTROL AUTOMATION AND SYSTEMS, 2019, 17 (03) :667-678
[6]   Carrier Phase-Based Precise Heading and Pitch Estimation Using a Low-Cost GNSS Receiver [J].
Ding, Wei ;
Sun, Wei ;
Gao, Yang ;
Wu, Jiaji .
REMOTE SENSING, 2021, 13 (18)
[7]   Bearing-only 2D maneuvering target tracking using smart interacting multiple model filter [J].
Ebrahimi, Mohsen ;
Ardeshiri, Mahdi ;
Khanghah, Sedigheh Alaie .
DIGITAL SIGNAL PROCESSING, 2022, 126
[8]   Vision meets robotics: The KITTI dataset [J].
Geiger, A. ;
Lenz, P. ;
Stiller, C. ;
Urtasun, R. .
INTERNATIONAL JOURNAL OF ROBOTICS RESEARCH, 2013, 32 (11) :1231-1237
[9]  
Geneva P, 2020, IEEE INT CONF ROBOT, P4666, DOI [10.1109/icra40945.2020.9196524, 10.1109/ICRA40945.2020.9196524]
[10]   Achieving Reliable Intervehicle Positioning Based on Redheffer Weighted Least Squares Model Under Multi-GNSS Outages [J].
Havyarimana, Vincent ;
Xiao, Zhu ;
Semong, Thabo ;
Bai, Jing ;
Chen, Hongyang ;
Jiao, Licheng .
IEEE TRANSACTIONS ON CYBERNETICS, 2023, 53 (02) :1039-1050