ElaMorph Projection: Deformation of 3D Shape by Dynamic Projection Mapping

We propose a projector-based method that provides an illusion of geometry change, similar to that caused by a change in physical properties such as elasticity, in response to inertia caused by physical motion. The proposed method is named "ElaMorph projection." Although several projection mapping methods capable of deforming targets have been previously proposed, these methods require the preparation of animations in advance. Moreover, these methods are unable to deform the shape in real-time according to actual movements of the object. To address these issues, we perform real-time geometry deformation and rendering based on the spatial motion of the object. To render the projection image, we extend the conventional method of deformation for 2D pictures or static 3D objects to include dynamic 3D objects. This study involves projection onto a dynamic 3D object; however, the projection quality decreases if a part of the rendered image extends beyond the projection target. To address this issue, the proposed algorithm ensures that the vertices after deformation always remain within the projection target. In addition, we develop a robust algorithm to generate projection images under dynamic illuminative conditions, through real-time estimation of the environmental lighting required for rendering. Moreover, using an elasticity map that can be easily constructed using a UV map, our method enables users to specify the vertices to be deformed, using an elasticity map. We present projections under several different sets of elasticity maps, environmental lighting, and elasticities. Finally, we evaluate the latency and throughput of our system.