Asynchronous iterative Q-learning based tracking control for nonlinear discrete-time multi-agent systems

This paper addresses the tracking control problem of nonlinear discrete-time multi-agent systems (MASs). First, a local neighborhood error system (LNES) is constructed. Then, a novel tracking algorithm based on asynchronous iterative Q-learning (AIQL) is developed, which can transform the tracking problem into the optimal regulation of LNES. The AIQL-based algorithm has two Q values Q(i)(A) and Q(i)(B) for each agent i , where Q(i)(A) is used for improving the control policy and Q(i)(B) is used for evaluating the value of the control policy. Moreover, the convergence of LNES is given. It is shown that the LNES converges to 0 and the tracking problem is solved. A neural network-based actor-critic framework is used to implement AIQL. The critic network of AIQL is composed of two neural networks, which are used for approximating Q(i)(A) and Q(i)(B) respectively. Finally, simulation results are given to verify the performance of the developed algorithm. It is shown that the AIQLbased tracking algorithm has a lower cost value and faster convergence speed than the IQL-based tracking algorithm.