Operation System for Simulation Roadheader Based on Visual Motion Capture

被引：0

作者：

Li, Yongling ^{[1
,3
,4
]}

Liu, Lingzhi ^{[1
,3
]}

Zhou, Baishun ^{[2
]}

Lei, Jingfa ^{[1
,3
,4
]}

Zhang, Miao ^{[1
,3
]}

Zhao, Ruhai ^{[1
,3
]}

机构：

[1] School of Mechanical and Electrical Engineering, Anhui Jianzhu University, Hefei

[2] School of Computer Science, China University of Labor Relations, Beijing

[3] Key Laboratory of Intelligent Manufacturing of Construction Machinery, Anhui Education Department, Hefei

[4] Sichuan Provincial Key Laboratory of Process Equipment and Control Engineering, Zigong

来源：

Xitong Fangzhen Xuebao / Journal of System Simulation | 2025年 / 37卷 / 06期

关键词：

approximate entropy; human-machine interaction; Kalman filter; motion capture; simulated operation system;

D O I：

10.16182/j.issn1004731x.joss.24-0516

中图分类号：

学科分类号：

摘要：

To enhance the natural human-machine interaction in simulation roadheader environment, a vision-based simulation roadheader operation system is proposed. The visual motion capture unit is based on the MediaPipe framework, which captures hand gestures through cameras and creates a correspondence between the physical world and virtual space. An improved Kalman filter algorithm is proposed by setting a weighted centroid to address the issue of unreasonable jumps in hand keypoint data during large-scale movements. The operator's gestures are discerned and the corresponding commands are conveyed. The results show that the improved method has significant advantages over the control group in terms of mean square error, signal-to-noise ratio, and approximate entropy parameters. The gesture recognition system is developed with an accuracy rate exceeding 92%. This interface enables the operator to efficiently control the simulated tunneling machine. © 2025 Acta Simulata Systematica Sinica. All rights reserved.

引用

页码：1531 / 1541

页数：10

共 28 条

[1]

Dong Zeyu, Liu Zheng, Li Yong, Et al., Operation Control System for Movable Robotic Arm Based on Mixed Reality, Laser & Optoelectronics Progress, 61, 4, pp. 271-279, (2024)

[2]

Shao Xuqiang, Zhang Haowei, Feng Xiaohua, Multi-sensory Fusion Method for Power Transformer Virtual Assembly, Journal of System Simulation, 34, 10, pp. 2244-2254, (2022)

[3]

Fang Wei, Xu Shuhong, Han Lei, Et al., Research and Application Progress of Tracking Registration Methods in AR Assembly, Journal of System Simulation, 35, 7, pp. 1438-1454, (2023)

[4]

Zhang Xuhui, Wang Tian, Zhang Chao, Et al., DT-driven Memory Cutting Control Method Using VR Instruction of Boom-type Roadheader, Journal of China Coal Society, 48, 11, pp. 4247-4260, (2023)

[5]

Zhang Xuhui, Zhang Chao, Wang Miaoyun, Et al., Digital Twin-driven Virtual Control Technology of Cantilever Roadheader, Computer Integrated Manufacturing Systems, 27, 6, pp. 1617-1628, (2021)

[6]

Cai Xingwang, Pei Yaoqiang, Yang Jihua, Et al., Research on Digital Twin Technology of Coal Mine Tunneling Machine System, Journal of System Simulation, 36, 10, pp. 2455-2468, (2024)

[7]

Wang Tao, Wu Yingnian, Yang Rui, Et al., Research on Real-time Gesture Classification Algorithm Based on IMU and sEMG Mixed Signals, Journal of System Simulation, 35, 2, pp. 359-371, (2023)

[8]

Zhu Yinlong, Shen Hongjun, Wu Jie, Et al., Data Glove Gesture Recognition Based on Flexible Strain Sensors, Transactions of the Chinese Society for Agricultural Machinery, 55, 6, pp. 451-458, (2024)

[9]

Wu Changcheng, Cao Qingqing, Fei Fei, Et al., Digital Gesture Recognition Method Based on Data Glove and Neural Networks, Journal of Southeast University (Natural Science Edition), 50, 3, pp. 563-569, (2020)

[10]

Ye Sufen, Lai Jizhou, Lu Pin, Et al., An Intelligent Perception and Gesture Recognition Technology for Wearable Piano-playing Glove, Chinese Journal of Scientific Instrument, 40, 5, pp. 187-194, (2019)

← 1 2 3 →