Diminished reality via multiple hand-held cameras

This paper proposes a novel method for calibrating multiple hand-held cameras target for Diminished Reality application. Our method does not require any special markers or information about camera parameters. Projective Grid Space (PGS) which is 3D space defined by epipolar geometry of two basis cameras is used for dynamic cameras calibration. Geometrical relations among cameras in PGS are obtained from 2D-2D corresponding points between views. We utilize Scale Invariant Feature Transform (SIFT) for finding corresponding points in natural scene for registering cameras to PGS. Moving object is segmented via graph cut optimization. Finally, the reconstructed visual hull is used to synthesize free viewpoint video in which unwanted or occluding object is deliberately removed. In the experimental results, free viewpoint video without unwanted object which is captured by hand-held cameras is successfully synthesized using the proposed method.