A simple trajectory optimization method with Q-learning for biped gait

Stable and intelligent gait trajectory generation is one of the important research issues in biped robot walking. This paper proposed a new simple optimization approach based on reinforcement learning method to achieve a both stable and reasonable trajectory. For a given robot with predefined rough gait, feasible actions were firstly taken on all joints at five key points in the gait to generate different kinds of trajectories, which were clustered later according to the ZMP stability criterion and required torques for learning. The most stable trajectory with feasible torque will finally be produced by using Q-learning method. According to the simulation results, learned trajectory has an obviously better motion curve merit than that before learning. And the corresponding ZMP trajectory approaches continuously toward the middle part of the stable region.