A Real-time Hand Gesture Recognition System on Raspberry Pi: A Deep Learning-based Approach

Recent years have witnessed the deep involvement of hand gesture recognition in Internet of Things (IoT) devices in healthcare, autonomous driving, virtual reality, augmented reality, etc., since hand gestures offer a natural and intuitive way for human beings to interact with IoT devices without relying on traditional input methods such as keyboards, mice, or touchscreens. Thus, accurate gesture recognition is crucial to its development. Consequently, many recognition approaches are proposed, from the traditional computer vision domain to the deep learning domain. Although with promising recognition performance, these approaches typically come with high computational and energy costs relying on graphics processing unit (GPU) accelerations, which cannot be compatible with low-cost and non-GPU IoT edge devices. To this end, in this paper, we develop an artificial intelligence (AI)-enabled system for real-time hand gesture recognition on low-cost edge devices, e.g., Raspberry Pis. Particularly, we first design a simple but effective convolutional neural network (CNN)-based model with residual blocks to handle American Sign Language (ASL) digits tasks, which enables fast-bust-accurate real-time inferences. Then, to fit the low-cost environment for edge devices, we involve model quantization techniques to shrink the model size, thus requiring reduced memory and storage for edge devices. In addition, we plug a motion sensor into our system to automatically turn off our recognition once it detects no people nearby, reducing energy consumption. By evaluating the performance of real-time hand gesture recognition, our system maintains a high prediction accuracy rate, e.g., over 96 %. Moreover, our designed solution has the potential to be privileged to other common low-cost edge devices.