Spatial-temporal dynamic hand gesture recognition via hybrid deep learning model

被引:1
作者
Jinghua Li
Huarui Huai
Junbin Gao
Dehui Kong
Lichun Wang
机构
[1] Beijing University of Technology,Beijing Key Laboratory of Multimedia and Intelligent Software Technology, Faculty of Information Technology
[2] The University of Sydney Business School,Discipline of Business Analytics
[3] The University of Sydney,undefined
来源
Journal on Multimodal User Interfaces | 2019年 / 13卷
关键词
Hand gesture recognition; Convolutional neural networks (CNN); Matrix variate restricted Boltzmann machine (MVRBM); Neural network (NN);
D O I
暂无
中图分类号
学科分类号
摘要
Hand gesture is a kind of natural interaction way and hand gesture recognition has recently become more and more popular in human–computer interaction. However, the complexity and variations of hand gesture like various illuminations, views, and self-structural characteristics make the hand gesture recognition still challengeable. How to design an appropriate feature representation and classifier are the core problems. To this end, this paper develops an expressive deep hybrid hand gesture recognition architecture called CNN-MVRBM-NN. The framework consists of three submodels. The CNN submodel automatically extracts frame-level spatial features, and the MVRBM submodel fuses spatial information over time for training higher level semantics inherent in hand gesture, while the NN submodel classifies hand gesture, which is initialized by MVRBM for second order data representation, and then such NN pre-trained by MVRBM can be fine-tuned by back propagation so as to be more discriminative. The experimental results on Cambridge Hand Gesture Data set show the proposed hybrid CNN-MVRBM-NN has obtained the state-of-the-art recognition performance.
引用
收藏
页码:363 / 371
页数:8
相关论文
共 50 条
  • [31] Hand gesture recognition using Leap Motion via deterministic learning
    Zeng, Wei
    Wang, Cong
    Wang, Qinghui
    MULTIMEDIA TOOLS AND APPLICATIONS, 2018, 77 (21) : 28185 - 28206
  • [32] Hand gesture recognition using Leap Motion via deterministic learning
    Wei Zeng
    Cong Wang
    Qinghui Wang
    Multimedia Tools and Applications, 2018, 77 : 28185 - 28206
  • [33] Real-Time Hand Gesture Recognition Model Using Deep Learning Techniques and EMG Signals
    Chung, Edison A.
    Benalcazar, Marco E.
    2019 27TH EUROPEAN SIGNAL PROCESSING CONFERENCE (EUSIPCO), 2019,
  • [34] Spatial--Temporal Synchronous Transformer for Skeleton-Based Hand Gesture Recognition
    Zhao, Dongdong
    Li, Hongli
    Yan, Shi
    IEEE TRANSACTIONS ON CIRCUITS AND SYSTEMS FOR VIDEO TECHNOLOGY, 2024, 34 (03) : 1403 - 1412
  • [35] HSFE Network and Fusion Model based Dynamic Hand Gesture Recognition
    Tai, Do Nhu
    Na, In Seop
    Kim, Soo Hyung
    KSII TRANSACTIONS ON INTERNET AND INFORMATION SYSTEMS, 2020, 14 (09) : 3924 - 3940
  • [36] Tiny Hand Gesture Recognition without Localization via a Deep Convolutional Network
    Bao, Peijun
    Maqueda, Ana I.
    del-Blanco, Carlos R.
    Garcia, Narciso
    IEEE TRANSACTIONS ON CONSUMER ELECTRONICS, 2017, 63 (03) : 251 - 257
  • [37] Hilbert sEMG data scanning for hand gesture recognition based on deep learning
    Tsinganos, Panagiotis
    Cornelis, Bruno
    Cornelis, Jan
    Jansen, Bart
    Skodras, Athanassios
    NEURAL COMPUTING & APPLICATIONS, 2021, 33 (07) : 2645 - 2666
  • [38] Hilbert sEMG data scanning for hand gesture recognition based on deep learning
    Panagiotis Tsinganos
    Bruno Cornelis
    Jan Cornelis
    Bart Jansen
    Athanassios Skodras
    Neural Computing and Applications, 2021, 33 : 2645 - 2666
  • [39] Deep Learning Enhanced Hand Gesture Recognition for Efficient Drone use in Agriculture
    Srinil, Phaitoon
    Thongnim, Pattharaporn
    INTERNATIONAL JOURNAL OF ADVANCED COMPUTER SCIENCE AND APPLICATIONS, 2024, 15 (05) : 1257 - 1264
  • [40] WiDG: An Air Hand Gesture Recognition System Based on CSI and Deep Learning
    Wang, Zhengjie
    Song, Xue
    Fan, Jingwen
    Chen, Fang
    Zhou, Naisheng
    Guo, Yinjing
    Chen, Da
    PROCEEDINGS OF THE 33RD CHINESE CONTROL AND DECISION CONFERENCE (CCDC 2021), 2021, : 1243 - 1248