Hand gesture teleoperation for dexterous manipulators in space station by using monocular hand motion capture

Space robots in the space station can assist or substitute astronauts to conduct some in-cabin tasks to alleviate the problems of heavy tasks and the limited number of astronauts in the space station. In response to these problems, a hand gesture teleoperation method is proposed for controlling of dexterous manipulators in the space station, which is based on monocular hand motion capture and video-calling tools. This method can realize free and convenient teleoperation. The work is mainly divided into two parts, which are monocular hand motion capture and teleoperation framework for dexterous manipulators. For monocular hand motion capture, a hand pose estimation method based on hand biological constraints from single RGB images is designed. On the other hand, a dexterous manipulator teleoperation framework based on hand motion capture and video -calling tools is designed. Experimental results carried out on hand pose estimation datasets proved that the proposed hand pose estimation method outperforms the state-of-the-art methods. Physical experiment on a space dexterous manipulator teleoperation platform validated the effectiveness of the proposed hand gesture teleoperation method.