Spatial Attention-Based 3D Graph Convolutional Neural Network for Sign Language Recognition

被引：14

作者：

Al-Hammadi, Muneer ^{[1
,2
]}

Bencherif, Mohamed A. ^{[1
,3
]}

Alsulaiman, Mansour ^{[1
,3
]}

Muhammad, Ghulam ^{[1
,3
]}

Mekhtiche, Mohamed Amine ^{[1
,3
]}

Abdul, Wadood ^{[1
,3
]}

Alohali, Yousef A. ^{[1
,4
]}

Alrayes, Tareq S. ^{[5
]}

Mathkour, Hassan ^{[1
,4
]}

Faisal, Mohammed ^{[1
,6
]}

Algabri, Mohammed ^{[1
,4
]}

Altaheri, Hamdi ^{[1
,3
]}

Alfakih, Taha ^{[1
]}

Ghaleb, Hamid ^{[1
,7
]}

机构：

[1] King Saud Univ, Ctr Smart Robot Res CS2R, Riyadh 11543, Saudi Arabia

[2] Norwegian Univ Sci & Technol, Dept Civil & Environm Engn, Fac Engn, Hogskoleringen 1, N-7034 Trondheim, Norway

[3] King Saud Univ, Dept Comp Engn, Coll Comp & Informat Sci, Riyadh 11543, Saudi Arabia

[4] King Saud Univ, Dept Comp Sci, Coll Comp & Informat Sci, Riyadh 11543, Saudi Arabia

[5] King Saud Univ, Dept Special Educ, Coll Educ, Riyadh 11543, Saudi Arabia

[6] Kuwait Coll Sci & Technol KCST, Ctr Robot, Kuwait 35004, Kuwait

[7] King Saud Univ, Software Engn Dept, Coll Comp & Informat Sci, Riyadh 11543, Saudi Arabia

来源：

SENSORS | 2022年 / 22卷 / 12期

关键词：

sign language recognition; graph convolutional neural network (GCN); attention; deep learning; HAND GESTURE RECOGNITION; FUSION;

D O I：

10.3390/s22124558

中图分类号：

O65 [分析化学];

学科分类号：

070302 ; 081704 ;

摘要：

Sign language is the main channel for hearing-impaired people to communicate with others. It is a visual language that conveys highly structured components of manual and non-manual parameters such that it needs a lot of effort to master by hearing people. Sign language recognition aims to facilitate this mastering difficulty and bridge the communication gap between hearing-impaired people and others. This study presents an efficient architecture for sign language recognition based on a convolutional graph neural network (GCN). The presented architecture consists of a few separable 3DGCN layers, which are enhanced by a spatial attention mechanism. The limited number of layers in the proposed architecture enables it to avoid the common over-smoothing problem in deep graph neural networks. Furthermore, the attention mechanism enhances the spatial context representation of the gestures. The proposed architecture is evaluated on different datasets and shows outstanding results.

引用

页数：15

共 47 条

[1] A survey on manual and non-manual sign language recognition for isolated and continuous sign [J].