3D motion trajectory analysis approach to improve Sign Language 3D-based content recognition

Current Sign Language animation technologies based on virtual reality cannot generate automatically animations in which the signer associates locations in space with entities under discussion. The automatic spatial association of signs locations is closely related to the sign recognition process. Without sign signature extraction through motion analysis, the spatial location of signs cannot be modified according to the change in location of entities under discussion. Currently, the generated signs animations are becoming common because they rely mainly on 3D-based content standard (X3D). Furthermore, the majority of studies rely on positions or rotations of virtual agent articulations as training data for classifiers or for matching techniques. Unfortunately, existing animation generation software use different 3D virtual agent content, therefore, articulation positions or rotations differ from system to other. Consequently, these methods are not efficient in the sign recognition process. In this paper, we propose a methodological foundation that aims to recognize signs through sign signature extraction, to be used to analyze automatically entities relations in the signing space and to change dynamically the sign location under discussion. Our recognition experiments were based on 900 ASL signs using Microsoft Kinect sensor to manipulate our X3D virtual agent. We have successfully recognized 887 isolated signs with 98.5 recognition rate and 0.3 second as recognition response time. These statistics allowed us to generate more than 200 phrases with automatic analysis and generation of spatial relations between different entities in the phrases. (C) 2012 Published by Elsevier B. V. Selection and/or peer-review under responsibility of Program Committee of INNS-WC 2012