Optimal model-free adaptive control based on reinforcement Q-Learning for solar thermal collector fields

This study addresses the challenge and related difficulties of controlling solar collector fields (SCFs) using high-complex models by proposing an adaptive optimal model-free controller based on the Reinforcement Q-Learning algorithm. The controller aims to achieve optimal performance using only plant measurements, particularly the working fluid temperature, flow rate, and reference trajectory. The proposed solution offers advantages over model-based controllers, as it can handle nonlinearities, time-varying model parameters, and computational costs associated with nonlinear models. Additionally, the Q-Learning algorithm requires a small amount of data, overcoming data-driven solutions presented in the literature till now for SCF systems. To ensure the controller performance, simulations are conducted using a validated SCF model based on actual data from the CIESOL thermal plant located in Almeria, Spain. The results demonstrate the effectiveness and usefulness of the model-free controller as the Q-Learning algorithm converges to the optimal gains of the Linear Quadratic Tracking (LQT) controller, exhibiting adaptive optimal response. Furthermore, the Q-Learning strategy outperforms the LQT controller in managing steady-state errors, particularly on sunny days with smooth solar irradiance variations. The average discrepancy in gains computed with reinforcement learning in a one-day campaign trial considering 18000 data is less than 15% compared to optimal LQT gains. This model-free approach proves its value and can easily be extended to different SCF systems without altering the general control formulation.