Optimizing Beam Hopping in Multibeam NGSO Constellations with Multi-Agent Reinforcement Learning

With the rapid development of Non-geostationary orbit (NGSO) satellite constellation, user demands have significantly increased. However, due to the high mobility of satellites and the uneven distribution of users within the coverage area, there are temporal and geographical variations in service demand. This leads to a situation where some beams in the system are unable to provide the required capacity, while others offer excess capacity, resulting in inefficient resource utilization. One of the primary challenges in future constellation design is addressing these imbalanced dynamic traffic demands. This paper proposes a novel approach to managing satellite resources focusing on beam hopping by leveraging multi-agent reinforcement learning. Compared with the traditional heuristic algorithm, the long-term return can be considered to better cope with the changing traffic demand. Moreover, the proposed agent reinforcement learning method has small action space, low training complexity and high training stability when dealing with large constellations. Simulation results demonstrate that proposed algorithm can effectively meet dynamically changing demands, thereby enhancing the overall service quality of the constellation.