Voltage control of DC-DC converters through direct control of power switches using reinforcement learning

It is well known that unmodeled dynamics and uncertainties can deteriorate the performance of classical controllers. To resolve this problem, there is growing popularity in using the capabilities of Artificial Intelligence (AI) algorithms, especially Reinforcement Learning (RL) in power systems, because it is a promising adaptive model-free control strategy that can take optimal decisions in unknown environments (dynamics). For this reason, in this paper, two state-of-the-art RL agents, namely Deep Q-Network (DQN) and Deep Deterministic Policy Gradient (DDPG), are used for voltage control of a DC-DC buck converter, and their performance is reported compared with other classical controllers such as Model Predictive Control (MPC) and Sliding Mode Control (SMC). The DQN agent directly controls the power switches of converters. In other words, based on the current condition of the converter, the agent decides whether or not to close the power switches. On the other hand, the DDPG agent and the other mentioned traditional controllers manipulate the duty cycle of a Pulse Width Modulation (PWM) signal to adjust the output voltage of the converter at desired setpoints. According to experimental results, both RL agents outperform the classical controllers in terms of transient response error and robustness against uncertainties. Also, with regard to computational costs and learning rate among RL-based controllers, the DQN agent can learn more from a single interaction with fewer computations because of its simpler structure and direct control of the switches of the converter. Additionally, one of the most important advantages of the RL-based controllers is that they can be applied to various configurations of DC-DC converters like buck, boost, and buck-boost converters, provided that it is retrained for the new environments. Finally, the number of transitions in the semiconductor switches of the converter reduces appreciably by using the DQN agent, which certainly prolongs their longevity.