Shared Reconstructed Environments in Extended Reality

Modern extended reality headsets can identify the geometry of the physical surrounding space, which can be shared and synchronized with remote users, allowing them to view the space through their own extended reality headsets from different points of view. Conversely, remote users can augment this shared space with additional virtual geometry. For instance, imagine wearing an augmented reality (AR) headset that identifies and shares your surrounding space with a remotely located interior designer. This interior designer can then virtually teleport into your environment and suggest furniture placements, which you can see in real-time through your AR glasses. Moreover, you can interact with each other via avatars representing each user. This research project demonstrates how two people in different locations can experience one's real physical environment in real-time and interact with each other. The strategies used to share geometry and user poses are discussed, along with the limitations of the hardware used for the prototype. The developed prototype involves one user wearing a Magic Leap 1 headset to scan their space, enabling a second user to remotely explore this space using an HTC Vive virtual reality (VR) headset. The VR user can also add virtual objects that both users can see.