Data-driven control of room temperature and bidirectional EV charging using deep reinforcement learning: Simulations and experiments

The control of modern buildings is a complex multi-loop problem due to the integration of renewable energy generation, storage devices, and electric vehicles (EVs). Additionally, it is a complex multi-criteria problem due to the need to optimize overall energy use while satisfying users' comfort. Both conventional rule-based (RB) controllers, which are difficult to apply in multi-loop settings, and advanced model-based controllers, which require an accurate building model, cannot fulfil the requirements of the building automation industry to solve this problem optimally at low development and commissioning costs. This work presents a fully data-driven pipeline to obtain an optimal control policy from historical building and weather data, thus avoiding the need for complex physics-based modelling. We demonstrate the potential of this method by jointly controlling a room temperature and an EV to minimize the cost of electricity while retaining the comfort of the occupants. We model the room temperature with a recurrent neural network and use it as a simulation environment to learn a deep reinforcement learning (DRL) control policy. It achieves on average 17% energy savings and 19% better comfort satisfaction than a standard RB room temperature controller. When a bidirectional EV is connected additionally and a two-tariff electricity pricing is applied, it successfully leverages the battery and decreases the overall cost of electricity. Finally, we deployed it on a real building, where it achieved up to 30% energy savings while maintaining similar comfort levels compared to a conventional RB room temperature controller.