Markerless Multi-View-based Multi-User Head Tracking System for Virtual Reality Applications

In this paper, we propose a markerless multiview-based multi-user head tracking system for VR/AR/MR applications. In the Virtual Reality domain, the user typically interacts with the virtual world through his movements. The point-of-view of the user's head mounted display (HMD) is based on the user's head pose. Most systems track the head of the user based on the tracking of the HMD itself and are typically constrained to a single user. Aiming to improve the user's experience in complex dynamic environments, we propose a head tracking method based on a multi-camera scenario, able to deal with the interaction of multiple users while coping with the existence of dynamic objects. The method relies on the voxelization of dense point cloud data, which is segmented to detect elements in the scene. Each element is holistically tracked along time and each detected user has his head's position tracked using a dedicated Particle Filter. The system was evaluated using an annotated ground-truth, according to several metrics, and demonstrated a mean 3D positioning error below 18 mm. Furthermore, tests with multiple users were performed that successfully validate the proposed head tracking approach. The aforementioned tests also showed that the proposed approach is computationally efficient (running time below 20 ms for 4 users).