Finger-STS: Combined Proximity and Tactile Sensing for Robotic Manipulation

This paper introduces and develops novel touch sensing technologies that enable robots to better sense and react to to intermittent contact interactions. We present Finger-STS, a robotic finger embodiment of the See-Through-your-Skin (STS) sensor that can capture 1) an "in the hand" visual perspective of an object that is being manipulated and 2) a high resolution tactile imprint of the contact geometry. We demonstrate the value of the sensor on a Bead Maze task. Here the multimodal feedback provided by the Finger-STS is leveraged by a robot to locate a bead visually and to guide it across a wire in response to tactile cues, with no additional sensing or planning required. To achieve this, we introduce a set of relevant visuotactile operations using computer vision-based algorithms. In particular, we sense the proximity of the object relative to the sensor as well as the nature of contact as a high resolution stick/slip vector field tracking the object motion in the finger.