Design and implementation of a soft Actor-Critic controller for a robotic arm

In this study, reinforcement learning (RL) was employed to train and control a robotic arm with six degrees of freedom in a point tracking task. The movements of such a robotic arm are not intuitive because they are determined by the rotations of and lengths between its six joints. Given this consideration, RL can be employed for the robotic arm to learn from its mistakes and autonomously explore the effects of joint rotation of its end point position. This approach is mostly used in complex learning tasks, such as those in gaming, robotics, and automated driving. The proposed learning process was verified in a simulator, in which point tracking and obstacle avoidance tasks were performed using RL algorithms. The performance of the soft actor-critic (SAC), proximal policy optimization, advantage actor-critic, and trust region policy optimization algorithms was compared in the point tracking task, and the results indicated that the SAC algorithm outperformed the other algorithms in this task. Therefore, the SAC algorithm was used with an adequately designed reward function to conduct model training, enabling the robotic arm to achieve obstacle avoidance in an object picking task. The established model was verified to allow the robotic arm to learn from failed experiences, enable fast navigation to the target position, and facilitate rapid deployment in future application.