Interference-Robust Millimeter-Wave Radar-Based Dynamic Hand Gesture Recognition Using 2-D CNN-Transformer Networks

Dynamic gesture recognition using millimeter-wave radar has a broad application prospect in the industrial Internet of Things (IoT) field. However, the existing methods in the random dynamic interference environment, such as throwing objects and waving and easily cause wrong recognition. This article proposes a dynamic gesture recognition method based on a convolutional neural network (CNN)-Transformer network to solve this problem. First, we reshape the original echoes acquired by the frequency-modulated continuous-wave (FMCW) millimeter-wave radar into 3-D data blocks in terms of Chirps x Samples x Frames. And we employ the mean elimination method to eliminate the static interference. Second, we extract dynamic gestures' distance and Doppler information with the 2-D fast Fourier transform and obtain the range-time map and Doppler-time maps. And we employ the coherent accumulation method to improve the signal-to-noise ratio (SNR). Third, we construct the CNN-Transformer network model for dynamic gesture recognition. The CNN is used to extract the local features of gestures, and multiple Transformer modules are stacked to extract deeper effective features. Finally, we build a data set for gesture recognition, including six kinds of dynamic gestures and two kinds of random interference. The experimental results show that the proposed method has a gesture recognition accuracy of more than 98% and 96% in the noninterference scene and the random dynamic interference scene, respectively, which are superior to the conventional recognition methods.