Adaptive optimal consensus of nonlinear multi-agent systems with unknown dynamics using off-policy integral reinforcement learning

Reinforcement learning (RL) has been identified as a promising approach for developing adaptive optimal consensus schemes for high-order strict-feedback nonlinear multi-agent systems (MASs). However, existing methods have limitations, as they can only be applied to systems with partially unknown dynamics and require an identifier-actor-critic framework. This paper proposes a novel approach that combines classical backstepping techniques and off-policy integral reinforcement learning (IRL) to circumvent these limitations and develop an adaptive optimal consensus scheme for nonlinear MASs with completely unknown dynamics. Specifically, we introduce an off-policy IRL-based adaptive optimal consensus scheme that can obtain optimal control inputs without knowledge of the system dynamics. The algorithm utilizes the actor-critic structure and updates the weight vectors using only one learning rule in each step based on the collected system trajectory data. We have proven that the optimal consensus is achieved, and the estimation errors of the optimal weight vectors are uniformly ultimately bounded (UUB). Finally, we present a simulation example to validate the effectiveness of the proposed approach.