A Split Sliding Window-Based Continuous Chinese Sign Language Recognition System

被引：0

作者：

Wang X.-Y. ^{[1
]}

Wang Q.-S. ^{[1
]}

Ma X.-D. ^{[1
]}

Liu P. ^{[2
]}

Dai H.-P. ^{[3
]}

机构：

[1] Institute of Mathematics, Hefei University of Technology, Hefei

[2] School of Computer Science, Hangzhou Dianzi University, Hangzhou

[3] School of Computer Science and Technology, Nanjing University, Nanjing

来源：

Beijing Youdian Daxue Xuebao/Journal of Beijing University of Posts and Telecommunications | 2021年 / 44卷 / 05期

关键词：

Bi-directional long short-term memory network; Data segmentation; Sign language recognition; Sliding window; Threshold;

D O I：

10.13190/j.jbupt.2021-001

中图分类号：

学科分类号：

摘要：

A large proportion of the world's disabled population is accounted for the individuals with hearing impairment which can communicate with people through the sign language. However, sign language is not mastered by the public, and there are still big obstacles between the individuals with hearing impairment and the normal people. A continuous Chinese sign language recognition system based on split sli-ding window (SSW) to realize automatic sign language recognition is proposed. The SSW system divides the sign language signal selected through the sliding window, and deletes one group of data to get new data in the original order, which is inputted to the sign language recognition neural network for training to obtain the gesture prediction value of a single sign language word. Finally, the majority voting strategy based on threshold is used to judge the identified prediction values. The SSW system is trained on 30 sign language sentences collected by 20 volunteers. The results show that the average accuracy of the SSW system reachs 83.9% on the test dataset, which is 16.7% higher than the long short-term memory model. © 2021, Editorial Department of Journal of Beijing University of Posts and Telecommunications. All right reserved.

引用

页码：48 / 54

页数：6

共 22 条

[1]

Mellon N, Niparko J K, Rathmann C, Et al., Should all deaf children learn sign language, Pediatrics, 136, 1, pp. 170-176, (2015)

[2]

Koller O, Zargaran S, Ney H, Et al., Deep sign: hybrid CNN-HMM for continuous sign language recognition, BMVC 2016, pp. 1-12, (2016)

[3]

Koller O, Zargaran S, Ney H, Et al., Re-sign: re-aligned end-to-end sequence modelling with deep recurrent CNN-HMMs, CVPR 2017, pp. 3416-3424, (2017)

[4]

Cui Runpeng, Liu Hu, Zhang Changshui, Et al., Recurrent convolutional neural networks for continuous sign language recognition by staged optimization, CVPR 2017, pp. 1610-1618, (2017)

[5]

Yang Wenwen, Tao Jinxu, Ye Zhongfu, Et al., Continuous sign language recognition using level building based on fast hidden Markov model, Pattern Recognition Letters, 78, 15, pp. 28-35, (2016)

[6]

Huang Jie, Zhou Wenggang, Zhang Qilin, Et al., Video-based sign language recognition without temporal segmentation, AAAI-18, pp. 2257-2264, (2018)

[7]

Guo Dan, Zhou Wengang, Wang Meng, Et al., Hierarchical LSTM for sign language translation, AAAI-18, pp. 6845-6852, (2018)

[8]

Zhang Jihai, Zhou Wengang, Li Houqiang, Et al., A threshold-based HMM-DTW approach for continuous sign language recognition, ICIMCS'14, pp. 237-240, (2014)

[9]

Li Kehuang, Zhou Zhengyu, Lee C H., Sign transition modeling and a scalable solution to continuous sign language recognition for real-world applications, ACM Transactions on Accessible Computing, 8, 2, pp. 1-23, (2016)

[10]

Bukhari J, Rehman M, Malik S I, Et al., American sign language translation through sensory glove

← 1 2 3 →