RGB-D Camera based 3D Human Mouth Detection and Tracking Towards Robotic Feeding Assistance

In this paper, an RGB-D camera based framework for the recognition and tracking of the human mouth for the purpose of autonomous robotic feeding is presented. The method employs the state-of-the-art face detection algorithm to acquire the 2D facial landmarks, and the corresponding 3D position of the human mouth is calculated using the depth information. In addition, a 3D point cloud visualizer of the human face with marked facial landmarks is provided for the purpose of visualization. The proposed system is applied in real-time vision-based robot control. Experiments indicate the validity of the proposed work in localising the mouth of different subjects served by the robot with a cup of water.