ANIMATED POINT CLOUDS REAL-TIME RENDERING FOR EXTENDED REALITY

Point cloud 3D models are becoming more and more popular thanks to the spreading of scanning systems employed in many fields, like autonomous vehicles and robotics. When used for rendering purposes, point clouds are usually displayed with their original color acquired at scan time, without considering the lighting condition of the scene where the model is placed. This leads to a lack of realism in many contexts, especially in the case of animated point clouds employed in eXtended Reality applications where it would be desirable to have the model reacting to incoming light and integrating with the surrounding environment. This paper proposes the application of Physically Based Rendering (PBR), a rendering technique widely used in Real-Time Computer Graphics applications, to animated point cloud models for reproducing specular reflections, and achieving a photo-realistic and physically accurate look under any lighting condition. Firstly, we consider the extension of commonly used animated point cloud formats, to include normal vectors, and PBR parameters (such as roughness and Metalness), as well as the encoding of the animated environment maps required by the technique. Then, an animated point cloud model is rendered with a shader implementing the proposed PBR method. Finally, the proposed PBR pipeline is compared to traditional renderings of the same point cloud obtained with commonly used shaders, under different lighting conditions in different environments. It will be shown how, using the proposed PBR method, the point cloud better integrates visually with its surroundings. Moreover, it will be shown that using this rendering technique it is possible to render different materials, by exploiting the features of PBR and the encoding of the surrounding environment.