Real-Time Object Detection and Recognition on Low-Compute Humanoid Robots using Deep Learning

We envision that in the near future, humanoid robots would share home space and assist us in our daily and routine activities through object manipulations. One of the fundamental technologies that needs to be developed for the robots is to enable them to detect objects and recognize them for effective manipulations and take real-time decisions involving the same. In this paper, we describe a novel architecture that enables multiple low-compute NAO robots to perform real-time detection, recognition and localization of objects in its camera view and take programmable actions based on the detected objects. The proposed algorithm for object detection and localization is an empirical modification of YOLOv3 along with a distributed architecture to operate multiple robots on a central "inference engine", based on indoor experiments in multiple scenarios, with a smaller weight size and lesser computational requirements. YOLOv3 was chosen after a comparative study of bounding box algorithms was performed with an objective to choose one that strikes the perfect balance among information retention, low inference time and high accuracy for real-time object detection and localization. Quantization of the weights and re-adjusting filter sizes and layer arrangements for convolutions improved the inference time for low-resolution images from the robot's camera feed. The architecture also comprises of an effective end-to-end pipeline to feed the real-time frames from the camera feed to the neural net and use its results for guiding the robot with customizable actions corresponding to the detected class labels.