Towards robust 3D reconstruction of human motion from monocular video

A robust, adaptive system for reconstructing 3D human motion from monocular video is presented. Our system takes a model based approach. To save computation time, we fit the skeleton instead of full body model to the silhouette. End sites positions identified from the silhouettes serve as an extra constraint. The alleviation of computational burden then makes the use of simulated annealing practical to get rid of local minima. The identifiable end sites also serve as the criterion to judge the reconstruction reliability of single frames. According to different reliabilities, the video is segmented into sections, which are reconstructed using different strategies. Our system is robust, getting rid of error accumulation in tracking, and adaptive, being able to tell the user when and where more information is needed.