Fractional-order Q-learning based on modal decomposition and convolutional neural networks for voltage control of smart grids

To maintain uniformity in the time scale of the control system, a coordinated first-level voltage control (FVC) framework is proposed for the voltage controller of three-state energy (TSE) models. To ensure control accuracy, this study combines complementary ensemble empirical mode decomposition (CEEMD) with adaptive noise (CEEMDAN), fractional-order proportional-integral-derivative (FO-PID) with convolutional neural networks (CNNs), Q-learning as CFOQL. Firstly, the CEEMDAN decomposes the historical voltage deviation data. The decomposed intrinsic mode functions (IMFs) are converted into RGB images with 30x30 pixels to train the CNNs. The residual function (RF) is utilized to train Q-learning. The implemented IMFs on the trained CNNs output, can generate regulating commands to adjust the differential and integral coefficients of FO-PID control. Similarly, the implemented RF on the trained Q-learning output can generate regulating commands. Then, the trained CNNs output the differential and integral coefficients for FO PID control; Q learning generates the regulating com mands based on the RF. Finally, the regulation commands generated by Q-learning are combined with the regulation commands generated by the FO-PID as the total regulation command. The proposed controller can effectively reduce the voltage deviations of smart grids (SGs) with higher control accuracy. Significantly, in the IEEE 2736-bus system, the error integration criterion indices obtained by the proposed controller are at least 7.53 %, 1.89 %, 13.80 %, and 14.61 % less than those of the proportional-integral-derivative (PID) algorithm, FOPID algorithm, R(lambda) algorithm, and Q(lambda) algorithm, respectively.