Reinforcement Learning DDPG-PPO Agent-Based Control System for Rotary Inverted Pendulum

The rotary inverted pendulum (RIP) system is a nonlinear system used as a benchmark for testing control strategies. RIP system has a lot of applications in balancing of robotic systems such as drones and humanoid robots. Controlling RIP system is a complex task without concise knowledge of classic control engineering. This paper uses the reinforcement learning (RL) approach to control the RIP instead of classical controllers such as PID (proportional-integral-derivative) and LQR (linear-quadratic regulator). In this work, the deep deterministic policy gradient-proximal policy optimization (DDPG-PPO) agent is proposed and implemented to control the rotary inverted pendulum platform both in simulation and hardware. DDPG agent with 13 layers is trained for the swing-up action of the pendulum, and the mode selection process is trained and tested using the PPO agent. The rotary inverted pendulum is controlled using a proposed controller and compared with various RL agents such as soft actor critic-proximal policy optimization (SAC-PPO). Additionally, the proposed method is tested with a conventional proportional-integral-derivative (PID) controller, for different pendulum mass values, to validate its effectiveness. Finally, the proposed RL controller is implemented on the real-time RIP apparatus (Quanser Qube-Servo). Results show that DDPG-PPO RL agent is much effective than SAC-PPO agent during swing-up control.