Optimal Scheduled Control Operation of Battery Energy Storage System using Model-Free Reinforcement Learning

Driven by the tremendous increase in rooftop solar panels and battery installations in Australian states, several studies have been conducted to efficiently manage battery operations with the imported grid power through battery energy storage systems (BESS). Therefore, it is crucial for the BESS to carefully decide the power set-points of the installed batteries to maintain user comfort while operating household appliances. Additionally, BESS should be capable of reducing the electricity bills by optimally managing the battery operation to sustain times of higher tariff prices for the imported grid power. This paper formulates the scheduled operation of the BESS as a Markov decision Process (MDP) that enables the BESS to figure out numerous scenarios and decides the optimal power set-points for both batteries and grid power. A model-free reinforcement learning approach is proposed to manage the batteries' powersharing and grid operation set-points to solve this MDP problem. This approach utilizes the advantages of the Deep Deterministic Policy Gradient (DDPG) algorithm to decide the shared power set-points every 5 minutes interval for the day ahead operation of the BESS. Finally, the proposed model is trained and validated using historical data of the Australian National Electricity market to offer an optimal scheduled control pattern for the daily BESS operations.