Collaborative optimization of multi-microgrids system with shared energy storage based on multi-agent stochastic game and reinforcement learning

Achieving the economical and stable operation of Multi-microgrids (MMG) systems is vital. However, there are still some challenging problems to be solved. Firstly, from the perspective of stable operation, it is necessary to minimize the energy fluctuation of the main grid. Secondly, the characteristics of energy conversion equipment need to be considered. Finally, privacy protection while reducing the operating cost of an MMG system is crucial. To address these challenges, a Data-driven strategy for MMG systems with Shared Energy Storage (SES) is proposed. In this paper, the Mixed-Attention is applied to fit the conditions of the equipment, and Multi-Agent Soft Actor-Critic(MA-SAC) , Multi-Agent Win or Learn Fast Policy Hill-Climbing (MA-WoLF-PHC) are proposed to solve the partially observable dynamic stochastic game problem. By testing the operation data of the MMG system in Northwest China, following conclusions are drawn: the R-Square (R2) values of results reach 0.999, indicating the neural network effectively models the nonlinear conditions. The proposed MMG system framework can reduce energy fluctuations in the main grid by 1746.5 kW in 24 h and achieve a cost reduction of 16.21% in the test. Finally, the superiority of the proposed algorithms is verified through their fast convergence speed and excellent optimization performance. •A data-driven method to describe the nonlinear conditions of energy conversion equipment was proposed.•A novel energy management-energy auction strategy based on MMG system framework with SES was designed.•SAC algorithm and WoLF-PHC algorithm were improved.•Privacy protection of MGs by nonlinear and non-convex constraints is realized.•The effect of different hyperparameters on the MA-SAC algorithm was analyzed.