Intelligent visual mouse system based on hand pose trajectory recognition in video sequences

Hand gesture recognition based on computer vision is considered as an efficient approach to establish communication between human and computer. This research provides a novel system to recognize dynamic hand gestures as different functions of mouse. For this purpose, a white glove is utilized in which fingertips have five different colors. Then, based on the functions of the mouse, 11 dynamic hand gestures are defined. In order to track the hand in each frame, the optical flow and GMM algorithms are used. Then, by using the mean and variance of the colors in each RGB plane, trajectories of the fingertips are detected. To make the acquired information richer, the Representative Trajectory of these five trajectories is computed. Features are extracted from the curves through a process inspired by the concept of shape context. In this process, each trajectory is normalized and then the histogram of extracted vector from normalized curve is calculated in a log-polar space. Thus, by this descriptor, a feature vector with length of 672 is created. Using the defined 11 dynamic gestures, a dataset including 220 observations is constructed and by using the aforementioned features, the relevant training matrix is formed. By employing PCA and Sequential Feature Selection techniques, dimension of the feature vector is reduced. For the experiment, different classifiers are applied and the experimental results confirm the privileged performance of the proposed system for the intelligent visual mouse.