Robust trajectory design and guidance for far-range rendezvous using reinforcement learning with safety and observability considerations

Observability, safety, and robustness are critical for successful rendezvous and proximity operation (RPO) missions. The use of angles-only navigation (AON) for these missions is often seen as limited due to its inability to determine range, though it remains appealing for its low cost. This work utilizes the proximal policy optimization algorithm in reinforcement learning for the guidance of the far-range phase of an RPO mission, ensuring observability, safety, and minimal fuel consumption under AON. Trajectory planning for the mission is done via a nonlinear optimizer, which also considers safety and observability. The constraint satisfaction challenges during trajectory planning and guidance are alleviated through the problem formulation, which incorporates Lambert's method to guarantee that the target state is always reached. During the training of the reinforcement learning controller, a predefined set of grid points in the initial state distribution is used to evaluate the policies and select the best policy fairly. The nominal and reinforcement learning-guided trajectories are validated for observability and safety, and the guidance controller's performance is tested through Monte Carlo simulations. Results show that for a 6 h mission previously presented in the literature, in the presence of errors, the reinforcement learning controller consumes 22.31% less Delta v compared to the next-best guidance strategy explored while fully adhering to safety and observability constraints.