A Comprehensive Study of 3-D Vision-Based Robot Manipulation

被引:33
|
作者
Cong, Yang [1 ,2 ,3 ]
Chen, Ronghan [1 ,2 ,3 ,4 ]
Ma, Bingtao [1 ,2 ,3 ,4 ]
Liu, Hongsen [5 ]
Hou, Dongdong [1 ,2 ,3 ,4 ]
Yang, Chenguang [6 ]
机构
[1] Chinese Acad Sci, Shenyang Inst Automat, State Key Lab Robot, Shenyang 110016, Peoples R China
[2] Chinese Acad Sci, Inst Robot, Shenyang 110169, Peoples R China
[3] Chinese Acad Sci, Inst Intelligent Mfg, Shenyang 110169, Peoples R China
[4] Univ Chinese Acad Sci, Beijing 100049, Peoples R China
[5] JD com Inc, Beijing 100176, Peoples R China
[6] South China Univ Technol, Sch Automat Sci & Engn, Guangzhou 510641, Peoples R China
关键词
Robots; Service robots; Grasping; Data acquisition; Pose estimation; Force; Cameras; 3-D object recognition; grasping estimation; motion planning; pose estimation; robot manipulation; 3D OBJECT RECOGNITION; POSE ESTIMATION; GRASP; SEGMENTATION; REGISTRATION; EXPLORATION; VERSATILE; ALGORITHM; FEATURES; DATASET;
D O I
10.1109/TCYB.2021.3108165
中图分类号
TP [自动化技术、计算机技术];
学科分类号
0812 ;
摘要
Robot manipulation, for example, pick-and-place manipulation, is broadly used for intelligent manufacturing with industrial robots, ocean engineering with underwater robots, service robots, or even healthcare with medical robots. Most traditional robot manipulations adopt 2-D vision systems with plane hypotheses and can only generate 3-DOF (degrees of freedom) pose accordingly. To mimic human intelligence and endow the robot with more flexible working capabilities, 3-D vision-based robot manipulation has been studied. However, this task is still challenging in the open world especially for general object recognition and pose estimation with occlusion in cluttered backgrounds and human-like flexible manipulation. In this article, we propose a comprehensive analysis of recent progress about the 3-D vision for robot manipulation, including 3-D data acquisition and representation, robot-vision calibration, 3-D object detection/recognition, 6-DOF pose estimation, grasping estimation, and motion planning. We then present some public datasets, evaluation criteria, comparisons, and challenges. Finally, the related application domains of robot manipulation are given, and some future directions and open problems are studied as well.
引用
收藏
页码:1682 / 1698
页数:17
相关论文
共 50 条
  • [41] Active Vision-Based Robot Localization and Navigation in a Visual Memory
    Mariottini, Gian Luca
    Roumeliotis, Stergios I.
    2011 IEEE INTERNATIONAL CONFERENCE ON ROBOTICS AND AUTOMATION (ICRA), 2011,
  • [42] Vision-Based Framework to Estimate Robot Configuration and Kinematic Constraints
    Ortenzi, Valerio
    Marturi, Naresh
    Mistry, Michael
    Kuo, Jeffrey
    Stolkin, Rustam
    IEEE-ASME TRANSACTIONS ON MECHATRONICS, 2018, 23 (05) : 2402 - 2412
  • [43] Monocular vision-based exploration with an outdoor nonholonomic mobile robot
    Alencastre-Miranda, M
    Muñoz-Gomez, L
    Swain-Oropeza, R
    PROCEEDINGS OF THE TENTH IASTED INTERNATIONAL CONFERENCE ON ROBOTICS AND APPLICATIONS, 2004, : 184 - 189
  • [44] A Comprehensive Review on Vision-Based Violence Detection in Surveillance Videos
    Ullah, Fath U. Min
    Obaidat, Mohammad S.
    Ullah, Amin
    Muhammad, Khan
    Hijji, Mohammad
    Baik, Sung Wook
    ACM COMPUTING SURVEYS, 2023, 55 (10)
  • [45] The MuseLearn Platform: Personalized Content for Museum Visitors Assisted by Vision-Based Recognition and 3D Pose Estimation of Exhibits
    Styliaras, G.
    Constantinopoulos, C.
    Panteleris, P.
    Michel, D.
    Pantzou, N.
    Papavasileiou, K.
    Tzortzi, K.
    Argyros, A.
    Kosmopoulos, D.
    ARTIFICIAL INTELLIGENCE APPLICATIONS AND INNOVATIONS, AIAI 2020, PT I, 2020, 583 : 439 - 451
  • [46] Research on Manipulation of Soft Tissue Based on 3D Vision
    Sun, Chengfeng
    Li, Peng
    Wu, Xiaojun
    Gao, Xing
    Liu, Yunhui
    2022 IEEE INTERNATIONAL CONFERENCE ON CYBORG AND BIONIC SYSTEMS, CBS, 2022, : 374 - 379
  • [47] Optimal Vision-Based Orientation Steering Control for a 3-D Printed Dexterous Snake-Like Manipulator to Assist Teleoperation
    Razjigaev, Andrew
    Pandey, Ajay
    Howard, David
    Roberts, Jonathan
    Jaiprakash, Anjali
    Crawford, Ross
    Wu, Liao
    IEEE-ASME TRANSACTIONS ON MECHATRONICS, 2024, 29 (02) : 1260 - 1271
  • [48] Vision-based dynamic modeling of wheeled-legged robot considering slippage using Gibbs-Appell formulation
    Korayem, M. H.
    Nikseresht, N.
    Asadi, A. H.
    ROBOTICA, 2024, : 3784 - 3802
  • [49] A Review on Viewpoints and Path Planning for UAV-Based 3-D Reconstruction
    Maboudi, Mehdi
    Homaei, MohammadReza
    Song, Soohwan
    Malihi, Shirin
    Saadatseresht, Mohammad
    Gerke, Markus
    IEEE JOURNAL OF SELECTED TOPICS IN APPLIED EARTH OBSERVATIONS AND REMOTE SENSING, 2023, 16 : 5026 - 5048
  • [50] Vision-based pose estimation of craniocervical region: experimental setup and saw bone-based study
    Zubair, Mohammad
    Kansal, Sachin
    Mukherjee, Sudipto
    ROBOTICA, 2022, 40 (06) : 2031 - 2046