A Cooperative Approach to Teleoperation Through Gestures for Multi-robot Systems

被引:0
|
作者
Martinelli, Dieisson [1 ]
Cerbaro, Jonathan [1 ]
Teixeira, Marco Antonio Simoes [3 ]
Kalempa, Vivian Cremer [2 ]
Monteiro, Vitor de Assis [2 ]
de Oliveira, Andre Schneider [1 ]
机构
[1] Fed Univ Technol Parana UTFPR, Ave Sete Setembro 3165, Curitiba, Parana, Brazil
[2] Univ Estado Santa Catarina UDESC, Luiz Fernando Hastreiter St 180, Sao Bento Do Sul, SC, Brazil
[3] Pontificia Univ Catolica Parana PUCPR, Grad Program Informat PPGIa, Rua Imaculada Conceicao 1155, Curitiba, Brazil
来源
SYNERGETIC COOPERATION BETWEEN ROBOTS AND HUMANS, VOL 2, CLAWAR 2023 | 2024年 / 811卷
关键词
Mobile robots; Teleoperation; Multi-robot systems; MediaPipe; Hands recognition;
D O I
10.1007/978-3-031-47272-5_18
中图分类号
TP [自动化技术、计算机技术];
学科分类号
0812 ;
摘要
A teleoperation system uses motion interface approaches that allow sending commands remotely to a mobile robotic base. Traditional equipment uses specific components and, many times, stocks on the operator's body, which may hinder their locomotion and teleoperation. In addition to being limited to teleoperation of only one robotic equipment, not being able to adapt to a multi-robot system. The multi-robot systems are generally used to increase the efficiency of the applications, having two or more robots working together to achieve a goal in a cooperative form, increasing the capacity and robustness of task accomplishments. In this sense, this paper presents the development of a cooperative approach to the teleoperation of multi-robot systems coupled with robot manipulators. This work uses the framework MediaPipe Hands to capture 2D points of the hand of the operator through RGB images combined with depth data through a Kinect sensor to get 3D points of the hand of the operator. These data are used to send positions that the mobile base and the robot manipulators must assume during the teleoperation. Experiments were conducted to demonstrate the algorithm's efficiency and precision during the realization of object pick-up tests. A video of the experiment was made and can be accessed through the link: https://youtu.be/10cVwcjPYOE.
引用
收藏
页码:211 / 223
页数:13
相关论文
共 50 条
  • [1] Gestures-teleoperation of a heterogeneous multi-robot system
    de Carvalho, Kevin Braathen
    Villa, Daniel Khede Dourado
    Sarcinelli-Filho, Mario
    Brandao, Alexandre Santos
    INTERNATIONAL JOURNAL OF ADVANCED MANUFACTURING TECHNOLOGY, 2022, 118 (5-6): : 1999 - 2015
  • [2] Gestures-teleoperation of a heterogeneous multi-robot system
    Kevin Braathen de Carvalho
    Daniel Khede Dourado Villa
    Mário Sarcinelli-Filho
    Alexandre Santos Brandão
    The International Journal of Advanced Manufacturing Technology, 2022, 118 : 1999 - 2015
  • [3] Teleoperation of Multi-Robot Systems to Relax Topological Constraints
    Sabattini, Lorenzo
    Capelli, Beatrice
    Fantuzzi, Cesare
    Secchi, Cristian
    2020 IEEE INTERNATIONAL CONFERENCE ON ROBOTICS AND AUTOMATION (ICRA), 2020, : 4558 - 4564
  • [4] Cooperative Heterogeneous Multi-Robot Systems: A Survey
    Rizk, Yara
    Awad, Mariette
    Tunstel, Edward W.
    ACM COMPUTING SURVEYS, 2019, 52 (02)
  • [5] A Visual Analytics Approach to Debugging Cooperative, Autonomous Multi-Robot Systems' Worldviews
    Bae, Suyun Sandrar
    Rossi, Federico
    Vander Hook, Joshua
    Davidoff, Scott
    Ma, Kwan-Liu
    2020 IEEE CONFERENCE ON VISUAL ANALYTICS SCIENCE AND TECHNOLOGY (VAST 2020), 2020, : 24 - 35
  • [6] Robust synchronisation of networked Lagrangian systems and its applications to multi-robot teleoperation
    Liu, Yen-Chen
    IET CONTROL THEORY AND APPLICATIONS, 2015, 9 (01): : 129 - 139
  • [7] Cooperative Flocking and Learning in Multi-Robot Systems for Predator Avoidance
    La, Hung Manh
    Lim, Ronny Salim
    Sheng, Weihua
    Chen, Jiming
    2013 IEEE 3RD ANNUAL INTERNATIONAL CONFERENCE ON CYBER TECHNOLOGY IN AUTOMATION, CONTROL AND INTELLIGENT SYSTEMS (CYBER), 2013, : 337 - +
  • [8] Cooperative exploration based on supervisory control of multi-robot systems
    Xuefeng Dai
    Laihao Jiang
    Yan Zhao
    Applied Intelligence, 2016, 45 : 18 - 29
  • [9] Cooperative exploration based on supervisory control of multi-robot systems
    Dai, Xuefeng
    Jiang, Laihao
    Zhao, Yan
    APPLIED INTELLIGENCE, 2016, 45 (01) : 18 - 29
  • [10] Object Manipulation Using Cooperative Mobile Multi-Robot Systems
    Mas, Ignacio
    Kitts, Christopher
    WORLD CONGRESS ON ENGINEERING AND COMPUTER SCIENCE, WCECS 2012, VOL I, 2012, : 324 - 329