Encountered-type visual haptic display using flexible sheet

An encountered-type visual haptic display using flexible sheet is proposed; it allows users to feel like seeing and pushing virtual soft objects directly. Both edges of a translucent flexible sheet such as rubber is attached to two manipulators. They apply bias tension to the sheet by pulling it from both sides, thus varying the sheet compliance. A user can feel the compliance of a virtual soft object by pushing the sheet with his finger directly. The motion of the finger tip is measured by stereo cameras. The manipulators also change the pose of the sheet along the object's surface together with the finger tip motion. The user can touch different points of the object and feel like stroking it with his finger. This sheet is also used as a rear projection screen. From the measured finger tip position, the deformation of the object is calculated by FEM. An LCD projector projects the CG image of the deformed object stereoscopically on the sheet from its back. The user can see the 3D image through stereoscopic glasses and touch the image directly. A method of correcting the CG image distortion caused by the movement of the sheet and the depression of the pushed sheet is proposed. A virtual soft cylinder is expressed by a prototype display: the stroking of the cylinder and the correction of its 3D image are evaluated.