Action recognition through fusion of sEMG and skeletal data in feature level

被引：2

作者：

Wang X. ^{[1
]}

Ding W. ^{[1
]}

Bian S. ^{[1
]}

Liu H. ^{[2
]}

机构：

[1] Department of Automation, Institute of Electrical Engineering, Key Laboratory of Intelligent Rehabilitation and Neuromodulation of Hebei Province, Yanshan University, 438 West of Hebei Avenue, Haigang District, Qinhuangdao

[2] School of Mechanical Engineering and Automation, State Key Laboratory of Robotics and Systems, Harbin Institute of Technology Shenzhen, Nanshan District, Shenzhen

来源：

Journal of Ambient Intelligence and Humanized Computing | 2022年 / 13卷 / 09期

基金：

中国国家自然科学基金;

关键词：

Action recognition; Feature extraction; Multimodal fusion;

D O I：

10.1007/s12652-022-03867-0

中图分类号：

学科分类号：

摘要：

Human action can be recognized through a unimodal way. However, the information obtained from a single mode is limited due to the fact that a single mode contains only one type of physical attribute. Therefore, it is motivational to improve the accuracy of actions through fusion of two different complementary modality, which are the surface electromyography (sEMG) and the skeletal data. In this paper, we propose a general framework of fusion of sEMG signals and skeletal data. Firstly, vector of locally aggregated descriptor (VLAD) was extracted from sEMG sequences and skeletal sequences, respectively. Secondly, features obtained from sEMG and skeletal data are mapped through different weighted kernels using multiple kernel learning. Finally, the classification results are obtained through the model of multiple kernel learning. A dataset of 18 types of human actions is collected via KinectV2 and Thalmic Myo armband to verify our ideas. The experimental results show that the accuracy of human action recognition are improved by combining skeletal data with sEMG signals. © 2022, The Author(s), under exclusive licence to Springer-Verlag GmbH Germany, part of Springer Nature.

引用

页码：4125 / 4134

页数：9

共 24 条

[1]

Bentley J.L., Multidimensional binary search trees used for associative searching, Commun ACM, 18, 9, pp. 509-517, (1975)

[2]

Chen C., Jafari R., Kehtarnavaz N., Improving human action recognition using fusion of depth camera and inertial sensors, IEEE Trans Hum Mach Syst, 45, 1, pp. 51-61, (2015)

[3]

Fusion of depth, skeleton, and inertial data for human action recognition, . In: 2016 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pp. 2712-2716, (2016)

[4]

Chen C., Jafari R., Kehtarnavaz N., A survey of depth and inertial sensor fusion for human action recognition, Multimedia Tools Appl, 76, 3, pp. 4405-4425, (2017)

[5]

Davis J.W., Bobick A.F., The recognition of human movement using temporal templates, IEEE Trans Pattern Anal Mach Intell, 23, 3, pp. 257-267, (2001)

[6]

) Focusing and diffusion: Bidirectional attentive graph convolutional networks for skeleton-based action recognition, . Arxiv Preprint Arxiv, 1912, (2019)

[7]

Guo Y., Lei L., Liu W., Cheng J., Tao D., Multiview Cauchy estimator feature embedding for depth and inertial sensor-based human action recognition, IEEE Trans Syst Man Cybern Syst, 47, 4, pp. 617-627, (2017)

[8]

Gonen M., Alpaydin E., Multiple kernel learning algorithms, J Mach Learn Res, 12, pp. 2211-2268, (2011)

[9]

Jegou H., Douze M., Schmid C., Perez P., Aggregating local descriptors into a compact image representation, Proc Cvpr, 238, 6, pp. 3304-3311, (2010)

[10]

Liu K., Chen C., Jafari R., Kehtarnavaz N., Fusion of inertial and depth sensor data for robust hand gesture recognition, IEEE Sens J, 14, 6, pp. 1898-1903, (2014)

← 1 2 3 →