Sign Language Recognition using micro-Doppler and Explainable Deep Learning

被引：1

作者：

McCleary, James ^{[1
]}

Garcia, Laura Parra ^{[1
]}

Ilioudis, Christos ^{[1
]}

Clemente, Carmine ^{[1
]}

机构：

[1] Univ Strathclyde, Dept Elect & Elect Engn, Glasgow, Lanark, Scotland

来源：

2021 IEEE RADAR CONFERENCE (RADARCONF21): RADAR ON THE MOVE | 2021年

关键词：

Detection and classification; British Sign Language (BSL) gesture recognition; Doppler radar; micro-Doppler signatures; convolution neural network (CNN); transfer learning;

D O I：

10.1109/RadarConf2147009.2021.9455257

中图分类号：

TM [电工技术]; TN [电子技术、通信技术];

学科分类号：

0808 ; 0809 ;

摘要：

In this paper, Sign Language Recognition and classification of the micro-Doppler signatures of different British Sign Language (BSL) gestures is studied. A database of four different BSL hand gesture motions is presented in the form of micro-Doppler signals, recorded with a continuous waveform radar. For detecting the presence of the micro-Doppler signatures, joint time-frequency is applied by calculating their spectrograms. Each individual gesture is expected to contain unique spectral characteristics that are exploited in order to classify the gestures. A deep learning approach with transfer learning is studied and discussed for carrying out the classification task. Following this, a novel explainable AI algorithm is implemented to give the user visual feedback, in the form of colour highlights, for the most relevant features used to classify each signal.

引用

页数：6

共 25 条

[1]

Ahmadi N, 2020, COMP STUDY STATE ART

[2]

Amin MG, 2018, IEEE RAD CONF, P1461, DOI 10.1109/RADAR.2018.8378780

[3]

Bragg D., 2019, 21 INT ACM SIGAC CES

[4]

Chen VC, 2011, ARTECH HSE RADAR LIB, P1

[5]

Deng J, 2009, PROC CVPR IEEE, P248, DOI 10.1109/CVPRW.2009.5206848

[6]

Erhan D., 2009, U MONTREAL, V1341, P1

[7]

Geoffrey EHinton., 2012, Improving neural networks by preventing co-adaptation of feature detectors

[8]

Gunning D, 2017, Def. Adv. Res. Proj. Agency, V2, P2, DOI [10.1126/scirobotics.aay7120, DOI 10.1126/SCIROBOTICS.AAY7120]

[9]

Gurbuz SZ, 2020, 2020 IEEE INTERNATIONAL RADAR CONFERENCE (RADAR), P232, DOI [10.1109/RADAR42522.2020.9114818, 10.1109/radar42522.2020.9114818]

[10] Pre-Trained AlexNet Architecture with Pyramid Pooling and Supervision for High Spatial Resolution Remote Sensing Image Scene Classification [J].

Han, Xiaobing ;

Zhong, Yanfei ;

Cao, Liqin ;

Zhang, Liangpei .

REMOTE SENSING, 2017, 9 (08)

← 1 2 3 →