A Static Sign Language Recognition Method Enhanced with Self-Attention Mechanisms

被引:0
|
作者
Wang, Yongxin [1 ]
Jiang, He [1 ]
Sun, Yutong [2 ]
Xu, Longqi [1 ]
机构
[1] Harbin Univ Sci & Technol, Sch Measurement & Control Technol & Commun Engn, Harbin 150080, Peoples R China
[2] Harbin Univ Sci & Technol, Sch Elect & Elect Engn, Harbin 150080, Peoples R China
关键词
static gesture recognition; high robustness; self-attention enhanced; CNN;
D O I
10.3390/s24216921
中图分类号
O65 [分析化学];
学科分类号
070302 ; 081704 ;
摘要
For the current wearable devices in the application of cross-diversified user groups, it is common to face the technical difficulties of static sign language recognition accuracy attenuation, weak anti-noise ability, and insufficient system robustness due to the differences in the use of users. This paper proposes a novel static sign language recognition method enhanced by a self-attention mechanism. The key features of sign language gesture classification are highlighted by the weight function, and then the self-attention mechanism is combined to pay more attention to the key features, and the convolutional neural network is used to extract the features and classify them, which realizes the accurate recognition of different types of static sign language under standard gestures and non-standard gestures. Experimental results reveal that the proposed method achieves an average accuracy of 99.52% in the standard static sign language recognition task when tested against the standard 36 static gestures selected within the reference American Sign Language dataset. By imposing random angular bias conditions of +/-(0 degrees-9 degrees] and +/-(9 degrees-18 degrees], the average recognition rates in this range were 98.63% and 86.33%. These findings indicate that, compared to existing methods, the proposed method not only maintains a high recognition rate for standard static gestures but also exhibits superior noise resistance and robustness, rendering it suitable for static sign language recognition among diverse user populations.
引用
收藏
页数:19
相关论文
共 50 条
  • [1] Context Matters: Self-Attention for Sign Language Recognition
    Slimane, Fares Ben
    Bouguessa, Mohamed
    2020 25TH INTERNATIONAL CONFERENCE ON PATTERN RECOGNITION (ICPR), 2021, : 7884 - 7891
  • [2] On the Combination of Multi-Input and Self-Attention for Sign Language Recognition
    Nam Vu Hoai
    Thuong Vu Van
    Dat Tran Anh
    INTERNATIONAL JOURNAL OF ADVANCED COMPUTER SCIENCE AND APPLICATIONS, 2024, 15 (04) : 1229 - 1235
  • [3] Local Context-aware Self-attention for Continuous Sign Language Recognition
    Zuo, Ronglai
    Mak, Brian
    INTERSPEECH 2022, 2022, : 4810 - 4814
  • [4] ESAformer: Enhanced Self-Attention for Automatic Speech Recognition
    Li, Junhua
    Duan, Zhikui
    Li, Shiren
    Yu, Xinmei
    Yang, Guangguang
    IEEE SIGNAL PROCESSING LETTERS, 2024, 31 : 471 - 475
  • [5] The Multimodal Scene Recognition Method Based on Self-Attention and Distillation
    Sun, Ning
    Xu, Wei
    Liu, Jixin
    Chai, Lei
    Sun, Haian
    IEEE MULTIMEDIA, 2024, 31 (04) : 25 - 36
  • [6] Adversarial Self-Attention for Language Understanding
    Wu, Hongqiu
    Ding, Ruixue
    Zhao, Hai
    Xie, Pengjun
    Huang, Fei
    Zhang, Min
    THIRTY-SEVENTH AAAI CONFERENCE ON ARTIFICIAL INTELLIGENCE, VOL 37 NO 11, 2023, : 13727 - 13735
  • [7] Isolated Sign Recognition from RGB Video using Pose Flow and Self-Attention
    De Coster, Mathieu
    Van Herreweghe, Mieke
    Dambre, Joni
    2021 IEEE/CVF CONFERENCE ON COMPUTER VISION AND PATTERN RECOGNITION WORKSHOPS, CVPRW 2021, 2021, : 3436 - 3445
  • [8] Self-attention for Speech Emotion Recognition
    Tarantino, Lorenzo
    Garner, Philip N.
    Lazaridis, Alexandros
    INTERSPEECH 2019, 2019, : 2578 - 2582
  • [9] A multi-modal fusion framework for continuous sign language recognition based on multi-layer self-attention mechanism
    Xue, Cuihong
    Yu, Ming
    Yan, Gang
    Qin, Mengxian
    Liu, Yuehao
    Jia, Jingli
    JOURNAL OF INTELLIGENT & FUZZY SYSTEMS, 2022, 43 (04) : 4303 - 4316
  • [10] Conversational Emotion Recognition Using Self-Attention Mechanisms and Graph Neural Networks
    Lian, Zheng
    Tao, Jianhua
    Liu, Bin
    Huang, Jian
    Yang, Zhanlei
    Li, Rongjun
    INTERSPEECH 2020, 2020, : 2347 - 2351