Multi-agent deep reinforcement learning based fully decentralized aggregation frequency regulation of electric vehicle

Electric vehicles (EVs) are high-quality flexible resources to provide frequency regulation services. However, the EV real-time participation in ancillary service market for frequency regulation (FRASM) under the existing aggregation framework faces data privacy and interest conflicts problems. To solve these issues, this paper proposes the fully-decentralized aggregator (FDA) as a trusted and non-profit agent to replace the traditional aggregator and help EVs interact with FRASM in a decentralized manner. Then a fully decentralized aggregation framework based on consortium blockchain is constructed to protect the interaction data security during frequency regulation process. Furthermore, the inference network is introduced so that EVs can learn better performance and optimize regulation mileage without sharing their private data. This paper proposes an inference network based multi-agent soft actor-critic (IN-MASAC) method that realizes the decentralized interaction between EVs and FRASM and considers multiple user preferences. The case studies demonstrate the efficiency and scalability of the fully decentralized aggregation framework and IN-MASAC method.