Action Temporal-Spatial Semantic Guide for 3D Human Pose Tracking

In this paper, a novel framework based on action temporal-spatial semantic guide for tracking 3D human poses from multi-view images is proposed. The inherent ambiguities caused by the lack of depth information or self-occlusion make it difficult to correctly track 3D human poses from 2D observations. To address this problem, we attempt to capture the important action temporal-spatial semantic knowledge to guide the update of rough 3D human pose hypotheses. The action semantic modeling is implemented by using both temporal and spatial restrictions. The action temporal restriction maintains time-consistency among sequential 3D human movements through a temporal manifold movement template while the action spatial restriction excavates movement correlation coefficient between human body limbs through linear partial least square regression algorithm. Extensive experiments on datasets of various human activities demonstrate that our proposed framework achieves the most significant pose tracking performance compare to the state-of-the-art methods without incorporating action temporal-spatial semantic guide.