Hand Gesture Recognition Based on Point Cloud Sequences and Inverse Kinematics Model

Hand gesture recognition is an important topic in human-computer interaction. With the rapid development of 3D sensors, gesture recognition methods using 3D input images have become mainstream. However, most of the current methods are based on depth maps and do not take full advantage of 3D information. In addition, static gesture recognition usually uses only one frame for recognition and cannot make use of redundant gesture-forming frames. This paper proposes a static gesture recognition method based on point cloud sequences and an inverse kinematics model. In the initial posture with five fingers open, the input point cloud sequences are divided into boundary points for fingertips marking and inner points for deformed joints marking, which is based on the K-curvature algorithm and curvature difference respectively. And those joints that cannot be marked by curvature differences, their positions are estimated by inverse kinematics. Then the bending angles and fingertips position are selected as features, and the recognition is completed by KNN. The performance of the proposed method has been experimentally evaluated using self-sampled data, and the average recognition accuracy is about 96%. Besides, because of the use of higher-order geometric features, the proposed method is highly abstract, easy to construct and adjust, and highly adaptable to different application scenarios.