Seeing in 3D: Assistive Robotics with Advanced Computer Vision

Robotics has made significant progress in cases of structured and constrained environments, e.g., manufacturing. However, it is still in its infancy when it comes to applications in unstructured and unconstrained situations e.g., social environments. In some respects, such as speed, strength and accuracy, robots have superior capacities compared to humans but that is not the case for person/object recognition, language, manual dexterity, and social interaction and understanding capabilities. Developing a computer vision system with Human visual recognition capabilities has been a very big challenge. It has been hindered mainly by: (i) the non-availability of 3D sensors (with the capabilities of the human eye) which are able to simultaneously capture appearance (colour and texture), surface shapes of objects while in motion, and (ii) the non-availability of algorithms to process this information in real-time. Recently, several affordable 3D sensors appeared in the market which is resulting in the development of practical 3D systems. Examples include 3D object and 3D face recognition for biometric applications, as well as the development of home robotic platforms to assist the elderly with mild cognitive impairment. The objective of the talk will be to describe few 3D computer vision projects and tools used towards the development of a platform for assistive robotics in messy living environments. Various systems with applications and their motivations will be described including 3D object recognition, 3D face/ear biometrics, grasping of unknown objects, and systems to estimate the 3D pose of a person.