Safety robustness of reinforcement learning policies: A view from robust control

For a reinforcement learning (RL) problem without a specified reward function, one may specify different reward functions to better guide an agent to learn. With different reward functions, the agent can learn different policies that generally have different robustness. Both the achieved reward and the success rate have been commonly used to evaluate the robustness of policies. Safety is a concern when using RL to solve problems in many safety-critical applications (e.g., robotic manipulation). However, evaluating the robustness of policies from the perspective of safety has not been discussed in the literature. The major contributions of this paper are the proposal of a novel concept of safety robustness to evaluate the robustness of policies from the perspective of safety and an algorithm to approximate the safety robustness of policies. To demonstrate how to implement the proposed algorithm, illustrative experi-ments are conducted and the safety robustness of three policies for controlling the manipulation of a cable-driven parallel robot is analyzed. Experiment results show that the proposed algorithm can approximate the safety robustness of policies using the ratio of the number of safe episodes to the number of total episodes and identify the best policy from multiple policies in terms of the safety of policies. (c) 2020 Elsevier B.V. All rights reserved.