Sign Language Recognition using micro-Doppler and Explainable Deep Learning

被引:1
作者
McCleary, James [1 ]
Garcia, Laura Parra [1 ]
Ilioudis, Christos [1 ]
Clemente, Carmine [1 ]
机构
[1] Univ Strathclyde, Dept Elect & Elect Engn, Glasgow, Lanark, Scotland
来源
2021 IEEE RADAR CONFERENCE (RADARCONF21): RADAR ON THE MOVE | 2021年
关键词
Detection and classification; British Sign Language (BSL) gesture recognition; Doppler radar; micro-Doppler signatures; convolution neural network (CNN); transfer learning;
D O I
10.1109/RadarConf2147009.2021.9455257
中图分类号
TM [电工技术]; TN [电子技术、通信技术];
学科分类号
0808 ; 0809 ;
摘要
In this paper, Sign Language Recognition and classification of the micro-Doppler signatures of different British Sign Language (BSL) gestures is studied. A database of four different BSL hand gesture motions is presented in the form of micro-Doppler signals, recorded with a continuous waveform radar. For detecting the presence of the micro-Doppler signatures, joint time-frequency is applied by calculating their spectrograms. Each individual gesture is expected to contain unique spectral characteristics that are exploited in order to classify the gestures. A deep learning approach with transfer learning is studied and discussed for carrying out the classification task. Following this, a novel explainable AI algorithm is implemented to give the user visual feedback, in the form of colour highlights, for the most relevant features used to classify each signal.
引用
收藏
页数:6
相关论文
共 25 条
[1]  
Ahmadi N, 2020, COMP STUDY STATE ART
[2]  
Amin MG, 2018, IEEE RAD CONF, P1461, DOI 10.1109/RADAR.2018.8378780
[3]  
Bragg D., 2019, 21 INT ACM SIGAC CES
[4]  
Chen VC, 2011, ARTECH HSE RADAR LIB, P1
[5]  
Deng J, 2009, PROC CVPR IEEE, P248, DOI 10.1109/CVPRW.2009.5206848
[6]  
Erhan D., 2009, U MONTREAL, V1341, P1
[7]  
Geoffrey EHinton., 2012, Improving neural networks by preventing co-adaptation of feature detectors
[8]  
Gunning D, 2017, Def. Adv. Res. Proj. Agency, V2, P2, DOI [10.1126/scirobotics.aay7120, DOI 10.1126/SCIROBOTICS.AAY7120]
[9]  
Gurbuz SZ, 2020, 2020 IEEE INTERNATIONAL RADAR CONFERENCE (RADAR), P232, DOI [10.1109/RADAR42522.2020.9114818, 10.1109/radar42522.2020.9114818]
[10]   Pre-Trained AlexNet Architecture with Pyramid Pooling and Supervision for High Spatial Resolution Remote Sensing Image Scene Classification [J].
Han, Xiaobing ;
Zhong, Yanfei ;
Cao, Liqin ;
Zhang, Liangpei .
REMOTE SENSING, 2017, 9 (08)