Dynamic Gesture Recognition Based on Deep 3D Natural Networks

被引:0
|
作者
Tie, Yun [1 ]
Zhang, Xunlei [1 ]
Chen, Jie [1 ]
Qi, Lin [1 ]
Tie, Jiessie [2 ]
机构
[1] Zhengzhou Univ, Sch Informat Engn, 100,Sci Ave, Zhengzhou 450001, Peoples R China
[2] Univ Toronto, Dept Comp Sci, Toronto, ON, Canada
关键词
Separable convolution; Long and short time memory network; Attention mechanism; 3D unequal frame feature extraction; Weight fusion strategy; NEURAL-NETWORK; HAND;
D O I
10.1007/s12559-023-10177-w
中图分类号
TP18 [人工智能理论];
学科分类号
081104 ; 0812 ; 0835 ; 1405 ;
摘要
The process of establishing contact with a computer through gestures still faces many bottlenecks that need to be solved, one of which is how to accurately recognize dynamic gestures. The traditional 3D convolution operation is transformed into a separable convolution operation based on separable convolution. The larger convolution kernel is transformed into smaller convolution and point convolution operations in the depth direction of the input data. The purpose of extracting and modeling the features of the input data is achieved, as well as the purpose of reducing the amount of calculation for the 3D convolution operation. Therefore, a long and short time memory network (which is more sensitive to temporal features) is introduced in the underlying network structure to memorize and model the relevant temporal information. Additionally, in order to overcome the adverse effects of the external background environment, an attention mechanism is also introduced into the structure to achieve suppression of the irrelevant background. On the basis of the existing research, by fusing the information of different modalities, the designed related experiments verify that the fusion operation is better than the recognition effect of single-modal information in dynamic gesture tasks. The experimental results show that three network structures, the efficient extraction of dynamic gesture features, and better recognition performance on the validation set of the IsoGD dataset can all be achieved. We propose a new method for dynamic gesture recognition, and the proposed model effectively improves the accuracy of gesture recognition.
引用
收藏
页码:2087 / 2100
页数:14
相关论文
共 50 条
  • [21] Fusion of 2D CNN and 3D DenseNet for Dynamic Gesture Recognition
    Zhang, Erhu
    Xue, Botao
    Cao, Fangzhou
    Duan, Jinghong
    Lin, Guangfeng
    Lei, Yifei
    ELECTRONICS, 2019, 8 (12)
  • [22] Hand gesture recognition via deep data optimization and 3D reconstruction
    Mustafa Z.
    Nsour H.
    ud din Tahir S.B.
    PeerJ Computer Science, 2023, 9
  • [23] Hand gesture recognition via deep data optimization and 3D reconstruction
    Mustafa, Zaid
    Nsour, Heba
    Tahir, Sheikh Badar ud din
    PEERJ COMPUTER SCIENCE, 2023, 9
  • [24] Real-time 3D pointing gesture recognition for natural HCI
    Guan, Yepeng
    Zheng, Mingen
    2008 7TH WORLD CONGRESS ON INTELLIGENT CONTROL AND AUTOMATION, VOLS 1-23, 2008, : 2433 - +
  • [25] Dynamic gesture recognition using 3D central difference separable residual LSTM coordinate attention networks☆
    Geng, Lichuan
    Chen, Jie
    Tie, Yun
    Qi, Lin
    Liang, Chengwu
    JOURNAL OF VISUAL COMMUNICATION AND IMAGE REPRESENTATION, 2025, 107
  • [26] Combined Dynamic Time Warping with Multiple Sensors for 3D Gesture Recognition
    Choi, Hyo-Rim
    Kim, TaeYong
    SENSORS, 2017, 17 (08):
  • [27] Heterogeneous hand gesture recognition using 3D dynamic skeletal data
    De Smedt, Quentin
    Wannous, Hazem
    Vandeborre, Jean-Philippe
    COMPUTER VISION AND IMAGE UNDERSTANDING, 2019, 181 : 60 - 72
  • [28] 3D separable convolutional neural network for dynamic hand gesture recognition
    Hu, Zhongxu
    Hu, Youmin
    Liu, Jie
    Wu, Bo
    Han, Dongmin
    Kurfess, Thomas
    NEUROCOMPUTING, 2018, 318 : 151 - 161
  • [29] Holoscopic 3D Micro-Gesture Recognition Based on Fast Preprocessing and Deep Learning Techniques
    Lei, Tao
    Jia, Xiaohong
    Zhang, Yuxiao
    Zhang, Yanning
    Su, Xuhui
    Liu, Shigang
    PROCEEDINGS 2018 13TH IEEE INTERNATIONAL CONFERENCE ON AUTOMATIC FACE & GESTURE RECOGNITION (FG 2018), 2018, : 795 - 801
  • [30] 3D Gesture Language Recognition System
    Chen, S. C.
    Chang, K. H.
    Liang, C. K.
    Lin, S. W.
    Huang, T. H.
    Hsieh, M. C.
    Yang, C. H.
    Wu, C. M.
    Lo, C. H.
    4TH KUALA LUMPUR INTERNATIONAL CONFERENCE ON BIOMEDICAL ENGINEERING 2008, VOLS 1 AND 2, 2008, 21 (1-2): : 773 - +