Augmented Reality Remote Collaboration with Dense Reconstruction

This paper describes an Augmented Reality remote collaboration system leveraging high-fidelity, dense scene reconstruction for intuitive and precise remote guidance. A local worker in need of help can use our system to automatically generate a 3D mesh of the surrounding and stream it to a remote expert. The remote expert can navigate and explore the reconstructed environment independently of the local worker in six degrees of freedom. World-stabilized text- and image-annotations can be placed in the scene and strokes drawn on surfaces are intelligently positioned in the world. In addition, the reconstruction allows the remote expert to segment colored objects from the mesh and use the resulting 3D model to create simple animations in order to convey precise instructions.