A novel approach for recovering the human body configuration based on the silhouette is presented. By considering pose inference as traversing the difference subspaces and using a data-driven mechanism, reversible jump Markov chain Monte Carlo (RJMCMC) can explore such solution space very efficiently. Experimental results are provided to demonstrate the efficiency and effectiveness of the proposed approach.