A photorealistic predictive display

The quality of a teleoperation system is decreased by time delays in the communication channel. Delays as low as a few hundred milliseconds between commanding an action and getting the visual feedback reduce the operator's performance. Predictive displays have proven their suitability to compensate for these delays, but at the expense of image quality when using computer-generated images. A photorealistic predictive display is presented that closes the feedback loop locally at the operator's side of a telepresence system. Photorealism is achieved using delayed camera images for texturing the predicted scene. Consumer graphics hardware is not only used for rendering but also for hardware-accelerated texture extraction. To allow concurrent access to model data, a multibuffer model structure is presented. A model of the teleoperator's environment is automatically acquired and updated by image processing techniques using a stereo camera as the only sensor.