Dynamic Hand Gesture Recognition using 2D Convolutional Neural Network

In recent years, many algorithms arise in the field of dynamic gesture recognition. Traditional methods lack of accuracy and rely heavily on hand-crafted features. Due to powerful ability of feature extraction, deep learning methods show amazing performance, especially Convolutional Neural Network (CNN) that has been addressed lately in video analysis. However, some CNN-based methods such as C3D and Two-Stream are time-consuming and a great deal of computation is needed. In this paper we propose a novel method for hand gesture recognition based on 2D CNN. For a given video sequence, the input of the network is no longer a number of sampled images. We encode each sampled image firstly and encoding vector of each image is just got. For each feature vector encoded, we stack them to generate a new image that contains rich spatio-temporal information of gesture. The new image instead of origin video is then sent in traditional 2D CNN model and the classification result of gesture is finally obtained. 3D spatio-temporal information has been compressed into 2D presentation in the course of classification, and the computation and time to be consumed are reduced. At the same time, it reduces the risk of overfitting to a certain extent. Based on proposed method, the performance is evaluated on the Microsoft Research 3D dataset (MSR3D). The experiment shows that our approach is highly effective and efficient at classifying a wide variety of actions on MSR3D.