Trajectory-Tracking Control of Robotic Systems via Deep Reinforcement Learning

This paper studies the trajectory tracking problems for a robotic manipulator and a mobile robot by using deep reinforcement learning based methods. A model-free deep reinforcement learning method based on Deep Deterministic Policy Gradient (DDPG) is designed for training. The priority replay memory is adopted to sample more significant transitions at each update. A distributed framework with multiple workers is proposed. Synchronous workers generate transitions and compute gradients for the global network, and collecting workers explore the environment with different policies and exploration noises. During the training, we adopt random reference slate initialization to solve the exploration problem, which can make the robots learn from the reference trajectory effectively. Numerical simulations are provided to demonstrate the effectiveness and efficiency of the proposed methods. It can be seen from the simulation results that the agent trained by the proposed distributed DDPG could learn faster and achieve smaller tracking errors than DDPG.