Serial-Parallel Dynamic Hand Gesture Recognition Network for Human-Robot Interaction

被引：2

作者：

Zhao, Yinan ^{[1
]}

Zhou, Jian ^{[1
]}

Ju, Zhaojie ^{[2
]}

Chen, Junkang ^{[3
]}

Gao, Qing ^{[3
]}

机构：

[1] Tongji Univ, Sch Mech Engn, Shanghai, Peoples R China

[2] Univ Portsmouth, Sch Comp, Portsmouth, Hants, England

[3] Sun Yat Sen Univ, Sch Elect & Commun Engn, Shenzhen, Peoples R China

来源：

2023 29TH INTERNATIONAL CONFERENCE ON MECHATRONICS AND MACHINE VISION IN PRACTICE, M2VIP 2023 | 2023年

基金：

中国国家自然科学基金;

关键词：

serial-parallel network; hand gesture recognition; human-robot interaction;

D O I：

10.1109/M2VIP58386.2023.10413398

中图分类号：

TP18 [人工智能理论];

学科分类号：

081104 ; 0812 ; 0835 ; 1405 ;

摘要：

In recent years, hand gesture recognition has played a crucial role in human-robot interaction (HRI). This paper proposes a skeleton-based serial-parallel dynamic hand gesture recognition network. A set of skeleton-based physical features is designed to model the spatial relationship of joints to construct skeletal space configurations. A slow-fast double-scale parallel network is proposed to extract the temporal dynamics of gestures. The attention mechanism is used to fuse the spatiotemporal information of the gestures, and the recognition result is obtained through the serial 1DCNN structure. In addition, the data enhancement technology based on transformation is used to improve the generalization of the network. The proposed methods are evaluated on the SHREC14 and SHREC28 datasets, which show superior performance, with an accuracy of 95.11% and 92.98%, respectively. The network is fine-tuned on the customized dataset HRIGes, and the recognition results are mapped to a five-fingered dexterous manipulator to realize real-time human-robot interaction.

引用

页数：6

共 21 条

[1]

Caputo FM., 2017, STAG, P9, DOI DOI 10.2312/STAG.20171221

[2] MFA-Net: Motion Feature Augmented Network for Dynamic Hand Gesture Recognition from Skeletal Data [J].

Chen, Xinghao ;

Wang, Guijin ;

Guo, Hengkai ;

Zhang, Cairong ;

Wang, Hang ;

Zhang, Li .

SENSORS, 2019, 19 (02)

[3] User-Independent Hand Gesture Recognition Classification Models Using Sensor Fusion [J].

Colli Alfaro, Jose Guillermo ;

Trejos, Ana Luisa .

SENSORS, 2022, 22 (04)

[4]

DeSmedt Q, 2017, P WORKSH 3D OBJ RETR, P33, DOI [10.2312/3dor.20171049, DOI 10.2312/3DOR.20171049]

[5]

Devineau G., 2018, PROC RECONNAISSANCE

[6] Hand gesture teleoperation for dexterous manipulators in space station by using monocular hand motion capture [J].

Gao, Qing ;

Li, Jinyang ;

Zhu, Yimin ;

Wang, Siyue ;

Liufu, Jingshu ;

Liu, Jinguo .

ACTA ASTRONAUTICA, 2023, 204 :630-639

[7] Dynamic Hand Gesture Recognition Based on 3D Hand Pose Estimation for Human-Robot Interaction [J].

Gao, Qing ;

Chen, Yongquan ;

Ju, Zhaojie ;

Liang, Yi .

IEEE SENSORS JOURNAL, 2022, 22 (18) :17421-17430

[8] Normalize d e dge convolutional networks for skeleton-based hand gesture recognition [J].

Guo, Fangtai ;

He, Zaixing ;

Zhang, Shuyou ;

Zhao, Xinyue ;

Fang, Jinhui ;

Tan, Jianrong .

PATTERN RECOGNITION, 2021, 118

[9]

Hou J., 2018, P EUROPEAN C COMPUTE, P0

[10] Temporal Decoupling Graph Convolutional Network for Skeleton-Based Gesture Recognition [J].

Liu, Jinfu ;

Wang, Xinshun ;

Wang, Can ;

Gao, Yuan ;

Liu, Mengyuan .

IEEE TRANSACTIONS ON MULTIMEDIA, 2024, 26 :811-823

← 1 2 3 →