Safe DQN-Based AoI-Minimal Task Offloading for UAV-Aided Edge Computing System

Utilizing the unmanned aerial vehicle (UAV) for task offloading over a large geographic area offers a promising solution to guarantee information freshness, i.e., Age of Information (AoI), in many of Internet of Things (IoT) applications. However, the energy limitations of both ground devices (GDs) and UAV wireless networks necessitate intelligent management of energy resources, as continuous energy consumption is involved in data sensing, transmission, and computation. Incorrect decision-making can exhaust the UAV's energy prematurely, endangering the efficacy of task offloading missions and potentially causing damage to the UAV itself. In this article, we investigate the problem of task offloading in an UAV-aided wireless powered edge computing system with a focus on enhancing information freshness while ensuring the UAV's energy-safety. To minimize the average AoI, we propose to jointly optimize GD wireless charging power, UAV flight trajectory, and offloading decisions. To prevent premature energy depletion in UAV operations, we formulate the optimization problem as a constrained Markov decision process (CMDP). Then, we introduce a novel safe deep Q-network (SDQN) algorithm, leveraging Lyapunov equations to derive an optimal strategy, which can strictly ensure that the actions of the UAV does not exceed its energy consumption limit. Through extensive simulations, we demonstrate the effectiveness of our proposed algorithm in minimizing AoI under energy consumption constraints.