Teleoperation Visualization System of Orchard Robot Based on Enhancing Telepresence

被引：0
作者：
Wang Y. [1 ]
Zhou J. [1 ]
Sun J. [1 ]
Wang K. [1 ]
Jiang Z. [1 ]
Zhang Z. [1 ]
机构：
[1] College of Engineering, Nanjing Agricultural University, Nanjing
来源：
Nongye Jixie Xuebao/Transactions of the Chinese Society for Agricultural Machinery | 2023年 / 54卷 / 03期
关键词：
3D reconstruction; augmented reality; orchard robot; teleoperation; telepresence;
D O I：
10.6041/j.issn.1000-1298.2023.03.002
中图分类号：
学科分类号：
摘要：
Aiming at the problem of lack of presence when the orchard robot used a monocular camera for remote operation, only using two-dimensional video to obtain environmental information lacked presence, a set of orchard environment information visualization system based on the enhanced sense of presence was designed. The system consisted of computing server, cloud server, network camera, LiDAR, embedded development platform, etc. The computing server adopted the T7920 workstation, and deployed the Tensorflow computing framework and the Open3D algorithm library of point cloud on it. After receiving the environmental image and point cloud data forwarded by the cloud server, the computing server enhanced the navigation information of the image, and surface reconstructed the point cloud. The embedded development platform could collect raw data from webcam and LiDAR, and uploaded them to cloud servers. A ZeroMQ-based message transfer program and HTML5 background service were deployed on the cloud server, providing cross-Internet message communication services and mobile teleoperation environment information visualization services. The test results showed that the average extraction time of the extraction model for navigation information deployed on the computing server was 86 ms, and the average precision of the navigation line extraction was 16°, which were better than the results of the comparison model. The algorithm of point cloud reconstruction can effectively establish scene contours with an average accuracy of 4. 9 cm and an average reconstruction time of 24 ms. The delay of compressed image transmission and enhancement processing did not exceed 230 ms, and the transmission delay of point cloud did not exceed 400 ms. The parameters could meet the basic requirements of the remote operation robot for orchard operation. The system significantly enhanced telepresence compared with that only with monocular cameras, which provided an effective reference for the remote operation of the orchard robot. © 2023 Chinese Society of Agricultural Machinery. All rights reserved.
引用
收藏
页码：22 / 31
页数：9
相关论文
共 32 条
  • [1] LIU Chengliang, GONG Liang, YUAN Jin, Et al., Current status and development trends of agricultural robots [J], Transactions of the Chinese Society for Agricultural Machinery, 53, 7, pp. 1-22, (2022)
  • [2] WANG Rujing, SUN Bingyu, Development status and expeetation of agricultural robot [J], Bulletin of Chinese Academy of Sciences, 30, 6, pp. 803-809, (2015)
  • [3] AMAYA A, ARACHCHIGE D D K, GREY J, Et al., Evaluation of human-robot teleoperation interfaces for soft robotic manipulators [C], Proceedings of the 2021 30th IEEE International Conference on Robot & Human Interactive Communication (ROMAN), (2021)
  • [4] ZHAO Xincan, ZHAN Penglei, WU Fei, Et al., Target recognition and registration in 3D dynamic scene based on domain knowledge[J], Robot, 40, 4, pp. 534-539, (2018)
  • [5] CHEN Yinglong, SONG Fujun, ZHANG Junhao, Et al., Telerobotic shared control strategy based on telepresence: a review, Journal of Zhejiang University (Engineering Science), 55, 5, pp. 831-842, (2021)
  • [6] LI Weihua, GUO Junlong, DING Liang, Et al., Review and prospect of ground teleoperation technology for lunar rover, Acta Aeronautica et Astronautica Sinica, 44, 1, pp. 163-182, (2023)
  • [7] ZHU Xiaojun, WANG Xueqian, MA Yunxuan, Et al., Design and experimental study on the teleoperation system for cable-driven and hyper-redundant space flexible manipulator [J], Robot, 44, 1, pp. 9-18, (2022)
  • [8] XUE Zhihui, LIU Jinguo, Review of space manipulator control technologies [J], Robot, 44, 1, pp. 107-128, (2022)
  • [9] YANG Xian, CHEN Chuanzhi, HUA Changchun, Stability analysis of teleoperated underwater manipulator system with contact force feedback, Control Theory & Applications, pp. 1-10
  • [10] QIN Liujie, SONG Guangming, MAO Juzheng, Et al., Shared control of multi-robot formations based on the eye-hand dual-modal human-robot interface [J], Robot, 44, 3, pp. 343-351, (2022)
1234