Multi-agent Reinforcement Learning-Based Energy Orchestrator for Cyber-Physical Systems

To reach a low-emission future it is necessary to change our behaviour and habits, and advances in embedded systems and artificial intelligence can help us. The smart building concept and energy management are key points to increase the use of renewable sources as opposed to fossil fuels. In addition, Cyber-Physical Systems (CPS) provide an abstraction of the management of services that allows the integration of both virtual and physical systems. In this paper, we propose to use Multi-Agent Reinforcement Learning (MARL) to model the CPS services control plane in a smart house, with the aim of minimising, by shifting or shutdown services, the use of non-renewable energy (fuel generator) by exploiting solar production and batteries. Moreover, our proposal is able to dynamically adapt its behaviour in real time according to the current and historical energy production, thus being able to address occasional changes in energy production due to meteorological phenomena or unexpected energy consumption. In order to evaluate our proposal, we have developed an open-source smart building energy simulator and deployed our use case. Finally several simulations are evaluated to verify the performance, showing that the reinforcement learning solution outperformed the heuristic-based solution in both power consumption and adaptability.