Deep Reinforcement Learning of Physically Simulated Character Control

Modeling the motion of humans and animals is a highly challenging problem in artificial intelligence. To Synthesize graceful and lifelike behaviors for physically simulated characters, traditional methods use motion capture, finite state machines, or morphology-specific knowledge to guide motion generation algorithms. While deep reinforcement learning offers broad avenues for synthesizing motion for simulated characters, the quality of synthesized motion in existing work often falls short of manually designed controllers and may exhibit significant artifacts. In this paper, we developed a simulated character control system based on deep reinforcement learning to address the challenges above. The system leverages a moderate amount of motion examples as prior knowledge. It employs reinforcement learning combined with deep neural networks to generate motions that are robust and much closer to a natural person's. Additionally, we use goal-directed reinforcement learning to guide the agent in performing user-specified tasks while imitating reference motion, such as walking to a designated location with a zombie-like gait. To minimize the occurrence of motion artifacts, we introduce the concept of motion mirror symmetry, encouraging symmetrical behavior in the agent by modifying the loss function. We demonstrate the effectiveness of our motion control system using a 3D humanoid robot, showing that our approach can produce symmetrical and lifelike behaviors.