Explainable AI and Robustness-Based Test and Evaluation of Reinforcement Learning

Reinforcement learning is a powerful and proven approach to generating near-optimal decision policies across domains, although characterizing performance boundaries, explaining decisions, and quantifying output uncertainties are major barriers to widespread adoption of reinforcement learning for real-time use. This is particularly true for high-risk and safety-critical aerospace systems where the cost of failure is high and performance envelopes for systems of interest may be small. To address these issues, this article presents a three-part test and evaluation framework for reinforcement learning, which is purpose-built from a systems engineering perspective on artificial intelligence. This framework employs explainable AI techniques-namely, Shapley additive explanations-to examine opaque decision-making, introduces robustness testing to characterize performance bounds and sensitivities, and incorporates output validation against accepted solutions. In this article, we consider an example problem of a high-speed aerospace vehicle emergency descent problem where a reinforcement learning agent is trained to control vehicle angle of attack (AoA). Shapley additive explanations expose the most significant features that impact the selection of AoA command while robustness testing characterizes the acceptable range of disturbances in flight parameters the trained vehicle can accommodate. Finally, the outputs from the reinforcement learning agent are compared with a baseline optimal trajectory as an acceptance criterion of RL solutions.