Reinforcement Learning-Based Tracking Control for Two Time-Scale Looper Hydraulic Servo Systems

Looper hydraulic servosystems (HSSs) are used in a variety of complex industrial environments. It is worth noting that there exists two-time-scale phenomenon in looper hydraulic servosystems. In this technical brief, the model of the looper hydraulic servosystems with two-time-scale phenomenon is established. In the framework of reinforcement learning, an optimal tracking control policy for looper hydraulic servosystems is proposed, which considers the effects of two-time-scale phenomena on the system. Then, by analyzing the change of looper angle in the actual industrial production process, the designed controller has a faster transition time compared to the proportional integral differential controller. In addition, the convergence of the optimal control solution obtained by using the proposed algorithm is proved. Finally, the effectiveness of the proposed method is verified by the simulation.