Safety-Critical Optimal Control of Discrete-Time Non-Linear Systems via Policy Iteration-Based Q-Learning

This paper investigates the problem of safety-critical optimal control for discrete-time non-linear systems. A safety-critical control algorithm is developed based on Q-learning and an iterative adaptive dynamic programming, that is, policy iteration. Discrete-time control barrier functions (CBFs) are introduced into the utility function for guaranteeing safety, in which a novel definition of the safe set and its boundary with multiple discrete-time CBFs are given. Also, for discrete-time systems, by using multiple discrete-time CBFs, the safety-critical optimal control problem of multiple safety objectives is addressed. Meanwhile, safety, convergence, and stability of the developed algorithm are rigorously demonstrated. An effective method to obtain an initial safety-admissible control law is established. Also, the developed algorithm is implemented by building an actor-critic structure with neural networks. Finally, the effectiveness of the proposed algorithm is illustrated by three simulation examples.