Reinforcement Learning for Energy-Efficient Trajectory Design of UAVs

Integrating unmanned aerial vehicles (UAVs) as aerial base stations (BSs) into terrestrial cellular networks has emerged as an effective solution to provide coverage and complement communication services in a fast and cost-effective manner. The three-dimensional (3-D) trajectories of UAVs have a remarkable impact on the performance of such networks. On the other hand, UAVs are battery limited, and thus optimizing their energy consumption is of high importance. In this regard, we propose a novel trajectory design mechanism for rotary-wing UAV-BSs in 3-D space to improve the energy efficiency of the network. In this approach, UAVs aim at maximizing an objective function that captures the tradeoff between energy consumption and throughput, while satisfying their ground users' quality-of-service requirements. Using reinforcement learning, we model our problem as a multiarmed bandit and propose an upper confidence bound-based algorithm to solve the problem. In our proposed mechanism, UAVs autonomously choose their velocities and update their locations adapted to the system conditions without requiring the prior and full knowledge of the system. Simulation results show that our proposed approach yields significant performance gains reaching up to 33.85% in terms of improving the network throughput, and up to 95% of enhancing the energy efficiency compared to a learning-based benchmark. Comparing to a nonlearning-based approach, our proposed approach improves the throughput and energy efficiency by 46.61% and 110%, respectively.