On the Benefit of FMG and EMG Sensor Fusion for Gesture Recognition Using Cross-Subject Validation

被引:0
作者
Rohr, Maurice [1 ]
Haidamous, Jad [1 ]
Schaefer, Niklas [2 ]
Schaumann, Stephan [2 ]
Latsch, Bastian [2 ]
Kupnik, Mario [2 ]
Antink, Christoph Hoog [1 ]
机构
[1] Tech Univ Darmstadt, AI Syst Med Lab, D-64283 Darmstadt, Germany
[2] Tech Univ Darmstadt, Measurement & Sensor Technol Grp, D-64283 Darmstadt, Germany
关键词
Electromyography; Muscles; Gesture recognition; Force; Hands; Accuracy; Sensor fusion; Data acquisition; Cameras; Feature extraction; Electromyography (EMG); ferroelectrets; force myography (FMG); gesture recognition; sensor fusion; FORCE MYOGRAPHY; SURFACE ELECTROMYOGRAPHY; CLASSIFICATION; PIEZOELECTRETS;
D O I
10.1109/TNSRE.2025.3543649
中图分类号
R318 [生物医学工程];
学科分类号
0831 ;
摘要
Hand gestures are a natural form of human communication, making gesture recognition a sensible approach for intuitive human-computer interaction. Wearable sensors on the forearm can be used to detect the muscle contractions that generate these gestures, but classification approaches relying on a single measured modality lack accuracy and robustness. In this work, we analyze sensor fusion of force myography (FMG) and electromyography (EMG) for gesture recognition. We employ piezoelectric FMG sensors based on ferroelectrets and a commercial EMG system in a user study with 13 participants to measure 66 distinct hand movements with 10ms labelling precision. Three classification tasks, namely flexion and extension, single finger, and all finger movement classification, are performed using common handcrafted features as input to machine learning classifiers. Subsequently, the evaluation covers the effectiveness of the sensor fusion using correlation analysis, classification performance based on leave-one-subject-out-cross-validation and 5x2cv-t-tests, and its effects of involuntary movements on classification. We find that sensor fusion leads to significant improvement (42% higher average recognition accuracy) on all three tasks and that both sensor modalities contain complementary information. Furthermore, we confirm this finding using reduced FMG and EMG sensor sets. This study reinforces the results of prior research about the effectiveness of sensor fusion by performing meticulous statistical analyses, thereby paving the way for multi-sensor gesture recognition in assistance systems.
引用
收藏
页码:935 / 944
页数:10
相关论文
共 50 条
  • [21] Controlling Mobile Robot Using IMU and EMG Sensor-based Gesture Recognition
    Shin, Seong-Og
    Kim, Donghan
    Seo, Yong-Ho
    2014 NINTH INTERNATIONAL CONFERENCE ON BROADBAND AND WIRELESS COMPUTING, COMMUNICATION AND APPLICATIONS (BWCCA), 2014, : 554 - 557
  • [22] Sensor Fusion Neural Networks for Gesture Recognition on Low-power Edge Devices
    Balazs, Gabor
    Chmurski, Mateusz
    Stechele, Walter
    Zubert, Mariusz
    ICAART: PROCEEDINGS OF THE 13TH INTERNATIONAL CONFERENCE ON AGENTS AND ARTIFICIAL INTELLIGENCE - VOL 2, 2021, : 141 - 150
  • [23] Comparing Cross-Subject Performance on Human Activities Recognition Using Learning Models
    Yang, Zhe
    Qu, Mengjie
    Pan, Yun
    Huan, Ruohong
    IEEE ACCESS, 2022, 10 : 95179 - 95196
  • [24] EMG Based Gesture Recognition Using Feature Calibration
    Kang, Kimoon
    Shin, Hyun-Chool
    2018 32ND INTERNATIONAL CONFERENCE ON INFORMATION NETWORKING (ICOIN), 2018, : 825 - 827
  • [25] Multi-Classifier Fusion Based on MI-SFFS for Cross-Subject Emotion Recognition
    Yang, Haihui
    Huang, Shiguo
    Guo, Shengwei
    Sun, Guobing
    ENTROPY, 2022, 24 (05)
  • [26] Cross-Subject and Cross-Modal Transfer for Generalized Abnormal Gait Pattern Recognition
    Gu, Xiao
    Guo, Yao
    Deligianni, Fani
    Lo, Benny
    Yang, Guang-Zhong
    IEEE TRANSACTIONS ON NEURAL NETWORKS AND LEARNING SYSTEMS, 2021, 32 (02) : 546 - 560
  • [27] Touch Gesture Recognition Using Spatiotemporal Fusion Features
    Li, Yun-Kai
    Meng, Qing-Hao
    Zhang, Hong-Wei
    IEEE SENSORS JOURNAL, 2022, 22 (01) : 428 - 437
  • [28] Cross-subject aesthetic preference recognition of Chinese dance posture using EEG
    Li, Jing
    Wu, Shen-rui
    Zhang, Xiang
    Luo, Tian-jian
    Li, Rui
    Zhao, Ying
    Liu, Bo
    Peng, Hua
    COGNITIVE NEURODYNAMICS, 2023, 17 (02) : 311 - 329
  • [29] A simplified adversarial architecture for cross-subject silent speech recognition using electromyography
    Cui, Qiang
    Zhang, Xingyu
    Zhang, Yakun
    Zheng, Changyan
    Xie, Liang
    Yan, Ye
    Wu, Edmond Q.
    Yin, Erwei
    JOURNAL OF NEURAL ENGINEERING, 2024, 21 (05)
  • [30] Cross-Subject emotion recognition from EEG using Convolutional Neural Networks
    Zhong, Xiaolong
    Yin, Zhong
    Zhang, Jianhua
    PROCEEDINGS OF THE 39TH CHINESE CONTROL CONFERENCE, 2020, : 7516 - 7521