Tracking hand and finger movements for behaviour analysis

In this paper, we describe ongoing work into methods for the automated tracking of hand and finger movements in interview situations. The aim of this work is to aid visual behaviour analysis in studies of deception detection. Existing techniques for tracking hand and finger movements are reviewed to place current and future work into context. Posterior probability maps of skin tone, based on Parzen colour space probability density estimates, are used for initial hand segmentation. Blob features are then used to produce a markup of hand-states. A complex wavelet decomposition, coupled to weightings provided by the posterior probability map, is applied to detect small hand and finger movements. We discuss our hand tracking algorithm based on blob feature extraction and the results obtained from motion and orientation parameters in a "high-stakes experiment", designed around a real-life situation. We suggest the role of kinematic models of upper body, limb and finger motion for future work. (c) 2006 Elsevier B.V. All rights reserved.