Deep Reinforcement Learning Based AoI Minimization for NOMA-Enabled Integrated Satellite-Terrestrial Networks

Integrated satellite-terrestrial networks (ISTN) are regarded as a critical 6G technology capable of providing seamless global communication coverage. However, to provide more real-time and efficient communication services, information freshness is becoming increasingly important in the ISTN. In this paper, we investigate a multi-agent DRL-based power control algorithm for minimizing the average age of information (AoI) in the non-orthogonal multiple access (NOMA) enabled ISTN. Specifically, a multi-agent proximal policy optimization (PPO) is employed to jointly allocate the power and reduce inter-beam interference. Due to the imperfect serial interference cancellation (SIC), the average AoI minimization is formulated as a non-convex problem that satisfies both quality of service constraints and total power limitations. Therefore, a generalized advantage estimation PPO-based power allocation (GAP3A) algorithm is proposed, which operates by centralized training and decentralized execution (CTDE) to enhance the training stability. Especially, an ingenious, global reward function is designed to guide the DRL training towards minimizing the average AoI of the ISTN. Simulation results demonstrate that the proposed CTDE-based GAP3A algorithm can effectively lower the average AoI to an optimal level and significantly outperforms the other three benchmarks.