Autonomous Decision-Making for Aerobraking via Parallel Randomized Deep Reinforcement Learning

Aerobraking is used to insert a spacecraft into a low orbit around a planet through many orbital passages into its complex atmosphere. The aerobraking atmospheric passages are challenging because of the high variability of the atmospheric environment. This paper develops a parallel domain randomized deep reinforcement learning architecture for autonomous decision-making in a stochastic environment, such as aerobraking atmospheric passages. In this context, the architecture is used for planning aerobraking maneuvers to avoid the occurrence of thermal violations during the atmospheric aerobraking passages and target a final low-altitude orbit. The parallel domain randomized deep reinforcement learning architecture is designed to account for large variability of the physical model, as well as uncertain conditions. Also, the parallel approach speeds up the training process for simulation-based applications, and domain randomization improves resultant policy generalization. This framework is applied to the 2001 Mars Odyssey aerobraking campaign; with respect to the 2001 Mars Odyssey mission flight data and a Numerical Predictor Corrector (NPC)-based state-of-the-art heuristic for autonomous aerobraking, the proposed architecture outperforms the state-of-the-art heuristic algorithm with a decrease of 97.5% in the number of thermal violations. Furthermore, it yields a reduction of 98.7% in the number of thermal violations with respect to the Mars Odyssey mission flight data and requires 13.9% fewer orbits. Results also show that the proposed architecture can also learn a generalized policy in the presence of strong uncertainties, such as aggressive atmospheric density perturbations, different atmospheric density models, and a different simulator maximum step size and error accuracy.