Trajectory Design for UAV-Based Inspection System: A Deep Reinforcement Learning Approach

In this paper, we consider a cellular connection-based UAV cruise detection system, where UAV needs traverse multiple fixed cruise points for aerial monitorning while maintain a satisfactory communication connectivity with cellular networks. We aim to minimize the weighted sum of UAV mission completion time and expected communication interruption duration by jointly optimizing the crossing strategy and UAV flight trajectory. Specifically, leveraging the state-of-the-art DRL algorithm, we utilize discrete-time techniques to transform the optimization problem into a Markov decision process (MDP) and propose an architecture with actor-critic based twin-delayed deep deterministic policy gradient(TD3) algorithm for aerial monitoring trajectory design (TD3-AM). The algorithm deals with continuous control problems with infinite state and action spaces. UAV can directly interacts with the environment to learn movement strategies and make continuous action values. Simulation results show that the algorithm has better performance than the baseline methods.