Minimizing AoI in High-Speed Railway Mobile Networks: DQN-Based Methods

This paper studies the high-speed railway mobile networks (HSRMN), where multiple railway-side sensors (RSs) are deployed along the track to sense environmental data, and multiple train-mounted sensors (TSs) are deployed on the train to collect train data. Both RSs and TSs are scheduled to transmit their sensed data respectively to the ground base station (BS) in a time division multiple access (TDMA) mode. To keep the data received at the BS from the RSs as fresh as possible and also ensure that the TSs complete the given uploading tasks, an optimization problem is established to minimize the average age of information (AoI) of the data gathered from RSs by jointly optimizing sensors' scheduling and transmission power control constrained by the maximum transmission power budget of RSs and TSs. Since the problem is non-convex and lacks an explicit expression of the objective function and the prior information about future channel state, we present a deep Q-learning network (DQN)-based method to solve it. Particularly, the BS is viewed as the agent, and the action space is constructed by scheduling policy and power control. To further accelerate the convergence speed of the presented DQN-based solution framework, an action space-reduced (ASR) version of the DQN-based method, i.e., the ASR-DQN-based method, is designed by deriving a closed-form solution to the optimal transmission power for a given sensors' scheduling policy. Numerical simulations show that, compared to the DQN-based method, the ASR-DQN-based method decreases the number of episodes required for convergence by about 23% and reduces the running time by about 41%. Moreover, compared with three baselines, i.e., the random method, the round-robin method, and the deep-Sarsa method, our presented ASR-DQN-based method achieves the lowest average AoI and has the best robustness among these compared methods.