Hand gesture/state recognition based on inertial measurement unit at high sample rate

In order to realize gesture recognition and hand state recognition at the same time, a single inertial measurement unit-based gesture recognition and touch recognition prototype was built, considering the inertial measurement unit at high sample rate has the capability of collecting motion signals and vibration signals simultaneously. The differences within hand state data and gesture data in the time and frequency domains were visually analyzed. Hand state, slipping gesture and circling gesture data sets were established. Considering the difference within data features, differential feature extraction methods were proposed, and neural network structures for hand state classification and gesture classification were constructed. Neural network models were trained by the data sets to achieve 99% accuracy rate in the comprehensive hand state recognition task, and 98% accuracy rate in both the slipping gesture recognition task and the circling gesture recognition task. A prototype program framework for real-time data stream processing, state shifting, and unknown class judgment was proposed. And a real-time program based on the hand state recognition model entities and the gesture recognition model entities was built, and the overall computational latency of the actual operation and the single model computational latency were measured, in order to prove the capability of real-time computing. Experimental results of model evaluation and real-time computing verification showed that, accurate and real-time hand states and gesture recognition with high sample rate inertial measurement units was feasible. © 2023 Zhejiang University. All rights reserved.