Joint Bidding Model of Electricity and Frequency Regulation Market With Wind Fire Storage Multi-agent Games Based on Improved Soft Actor-critic

For the power systems with high wind power penetration, we propose an improved Soft Actor-Critic joint bidding model for the electricity and frequency regulation market considering a multi-agent stochastic game of wind, fire, and storage. Firstly, a day-ahead (DA) bidding model for the electricity and frequency regulation auxiliary services market considering the opportunity cost of the frequency regulation in different wind speed zones is constructed to overcome the difficulty in weighing the benefits of the wind power producers participating in the electricity and frequency regulation markets at different wind speeds. Secondly, to address the wind power abandonment due to the risks of the deviation penalty avoidance, a new wind power deviation P2P trading market is established by combining the deviation of the wind power DA winning capacity with the actual intra-day power output, and a bilateral wind-fire bidding model considering the adaptive price constraints is designed to conduct the gaming bidding transactions before the start of the real-time (RT) balancing market. Finally, to address the complex game relationships among the subjects in the proposed model and the sample duplications of the deep reinforcement learning bidding method, a new Soft Actor-Critic bidding method based on the sample rate reduction is proposed, which avoids the discretion of the bidding strategies and the local optimum of the gradient optimization of the deterministic policy. The sample reduction reduces the training time for each market agent to obtain an optimized bidding strategy. The simulation analysis verifies the applicability and effectiveness of the model and method proposed in this paper. © 2023 Power System Technology Press. All rights reserved.