Energy-Optimal Trajectory Planning for Near-Space Solar-Powered UAV Based on Hierarchical Reinforcement Learning

One of the key technologies for achieving day and night flight, tracking solar peak, and reducing flight energy consumption for a near-space solar-powered unmanned aerial vehicle (UAV) is trajectory planning. However, the environmental differences faced by the near-space solar-powered UAV during long-term flight pose challenges to its online trajectory planning. This article introduces a hierarchical guidance method designed using a hierarchical reinforcement learning algorithm, which includes a two-layer neural network structure of bottom-level trajectory planning models and a top-level decision model. The top-level decision maker selects the appropriate bottom-level planner based on flight and current environmental information, while the planner outputs thrust, attack angle, and bank angle commands based on the input information. This hierarchical guidance structure can improve the UAV's adaptability to energy environment variations and realize an autonomous flight based on energy maximization in long-term missions. Flight simulations spanning spring, summer and autumn seasons show that the guidance controller is able to switch flight policies on its own as the environment changes, allowing the UAV to maximize energy gain on each day, thereby achieving the best energy management strategy in long-term flight. The simulation results also verify the over-fitting and under-fitting effects of the neural network in the solar UAV trajectory planning task, providing support for the necessity of hierarchical guidance.