Model-Free Algorithms for Containment Control of Saturated Discrete-Time Multiagent Systems via Q-Learning Method

In this article, we propose two model-free algorithms using state or output feedback for saturated discrete-time multiagent systems (SDTMASs) to attain global containment control. In most previous works, the control input can avoid saturation by utilizing the low gain feedback (LGF) method whereas requiring the knowledge of agent dynamics, and SDTMASs just can attain semi-global containment control. Distinct with the previous works, first, based on the Q-learning (QL) technique, this article defines a Q-function and deduces the corresponding QL Bellman equation, which is the most important part of the QL algorithm. Then, in order to solve the QL Bellman equation, we propose two iterative model-free algorithms using state and output feedback, and the LGF matrix can be acquired from that solution directly. Furthermore, under the state and output feedback control protocols with the feedback matrices obtained from the proposed model-free algorithms, the SDTMASs can achieve global containment control instead of semi-global containment control. Finally, we present some simulations to confirm the validity of the proposed algorithms.