A Distributed Actor-Critic Learning Approach for Affine Formation Control of Multi-Robots With Unknown Dynamics

Formation maneuverability is particularly important for multi-robots (MRs), especially when the robots are operating cooperatively in complex and dynamic environments. Although various methods have been developed for affine formation, it is still a difficult problem to design an affine formation controller for MRs with unknown dynamics. In this paper, a distributed actor-critic learning approach (DACL) in a look-ahead rollout manner is proposed for the affine formation of MRs under local communication, which improves the online learning efficiency. In the proposed approach, a distributed data-driven online optimization mechanism is designed via the sparse kernel technique to solve the near-optimal affine formation control issue of MRs with unknown dynamics as well as improve control performance. The unknown dynamics of MRs are learned offline based on precollected input-output datasets, and the sparse kernel-based approach is employed to increase the feature representation capability of the samples. Then, the proposed distributed online actor-critic algorithm for each robot in the formation includes two neural networks, which are utilized to approximate the costate functions and the near-optimal policies. Moreover, the convergence analysis of the proposed approach has been conducted. Finally, numerical simulation and KKSwarm-based experiment studies are performed to verify the effectiveness of the proposed approach.