Incorporating Observation Uncertainty into Reinforcement Learning-Based Spacecraft Guidance Schemes

In this paper, reinforcement learning is used to train a policy that successfully executes a multi-impulse spacecraft orbit transfer between periodic orbits in the Earth-Moon circular restricted three-body problem. A methodology is then developed to make the policy more robust to uncertainties in the observation state (i.e. navigation uncertainties). These methods are applied to a L-2 Lyapunov-to-halo orbit transfer in the Earth-Moon circular restricted three-body problem. This paper demonstrates that a reinforcement learning-based maneuver planner becomes more robust to uncertainties in the state observations by both including these uncertainties in the training environment, and by augmenting the observation vector with knowledge of the state uncertainty magnitudes.