Surface Prediction for Spatial Augmented Reality Applications

In spatial augmented reality applications, incorrect projection mapping may occur when projecting images onto moving non-rigid surfaces. This may detract from the user experience, as the image may not be perceived as originally intended. This is especially apparent when using low-cost projectors and cameras or when surfaces are moving quickly. In this paper, an algorithm is developed which predicts the motion of a non-rigid surface, so that when an image is being projected onto the surface, the projection "matches" the surface shape, while using low-cost equipment. Using an interconnected mass-spring-damper system to model the surface, the surface position is predicted using a Kalman filter-based algorithm, which also compensates for the processing delays and fast-moving surfaces. To accurately model real-world materials, the mass-spring system parameters are found using a system identification approach. When the prediction algorithm is implemented experimentally, in real-time, the results show convergent results in the sense that the surface predictions converge to the measured position of a non-rigid surface. The error results show that the algorithm is both accurate and robust and can currently be applied in spatial augmented reality applications.