HRC of intelligent assembly system based on multimodal gesture control

被引:1
|
作者
Duan, Jianguo [1 ]
Fang, Yuan [2 ]
Zhang, Qinglei [1 ]
Qin, Jiyun [1 ]
机构
[1] Shanghai Maritime Univ, China Inst FTZ Supply Chain, Shanghai, Peoples R China
[2] Shanghai Maritime Univ, Logist Engn Coll, Shanghai, Peoples R China
来源
INTERNATIONAL JOURNAL OF ADVANCED MANUFACTURING TECHNOLOGY | 2023年 / 127卷 / 9-10期
关键词
HRC; Gesture control; Multimodal perception; Integrally shrouded blade; RECOGNITION;
D O I
10.1007/s00170-023-11804-4
中图分类号
TP [自动化技术、计算机技术];
学科分类号
0812 ;
摘要
As a natural expression of the human body, gestures are widely used in robot control. However, most gesture control methods rely only on single-modal feature representation, which has certain instability. To this end, this paper proposes a human-robot collaboration (HRC) framework for intelligent assembly systems with multimodal gesture control. The framework aims to perform gesture recognition on RGB data and RGB-D data through convolutional neural networks (CNN) and combine gesture data from both modalities for application in robot control, enabling data fusion and feature sharing. Perform asynchronous integration of image acquisition and gesture detection with spatially and time-aligned RGB frames and RGB-D frames to ensure real-time detection of gestures, and speed and separation monitoring to ensure the safety of the collaboration process. To verify the effectiveness of this framework, it was applied to the assembly scene of the integrally shrouded blade-rotor system. Experiments show that the HRC of the intelligent assembly system with multimodal gesture control can better realize the human-robot collaborative assembly with the integrally shrouded blade and improve the intelligence of the assembly process.
引用
收藏
页码:4307 / 4319
页数:13
相关论文
empty
未找到相关数据