Synchronization of Motion from Multiple Humans and Humanoid Robot based on Kinect V2

Humanoid robots have evolved into a key technology required for many applications which are beyond the reach of humans. As these robots have high number degrees of freedom, a conventional control system is hard to develop. Tele-operation of these humanoid robots by humans is one area which has drawn a lot of interest from the researchers in recent times. This paper proposes a method to imitate the physical movements of multiple persons using Kinect V2 developed by Microsoft. Two main issues have been addressed in this paper. One is finding out the position of speaker when a verbal command is given and the other is to track the body of the speaker and mimic. Multi microphone array in Kinect V2 has been used for speech recognition and to point the direction of the speaker. Once a mimic command is given by a person, the location of his/her body is obtained and the skeletal data of this person would be used to imitate. The skeletal data of the person being tracked will contain 25 three dimensional coordinates which are the positions of various joints in the body with respect to Kinect V2. Angles at various joints of the skeleton are extracted and given to the microcontroller which controls the humanoid robot. Humanoid robots are not very much stable, whenever the posture of the robot changes there is every chance that the robot may go out of balance. Hence a Zero Moment Point (ZMP) based balance algorithm is implemented for the balancing of the robot.