LARR: A Localization-Assisted Method to Conceal Latency-Induced Position Errors in MR Remote Rendering

The future of social interaction lies in the Metaverse, where mixed reality (MR) plays a key role. With the growth of the Metaverse, there is a growing demand for volumetric videos in the media industry. However, rendering complex volumetric content on mobile devices with limited computing power presents a challenge. To address this, we propose a localization-assisted remote rendering (LARR) architecture for MR applications, which can effectively conceal delay-induced position error and display streamed objects accurately. The LARR architecture uses dedicated object streaming cameras (OSC) at the server for each 3D object to enable real-time position updates. In addition, stream data optimization is used to minimize unnecessary data transfer. The LARR approach reduces streaming data by 81% and 46% compared to non-optimized and existing MR streaming architectures in real-world experiments.