Limited Information Aggregation for Collaborative Driving in Multi-Agent Autonomous Vehicles

Multi-agent reinforcement learning (MARL) methods have emerged as a promising solution for multi-agent collaborative driving in the intersection and roundabout scenarios. However, these methods need large amounts of training data obtained from the interaction with the driving simulator, and learning from limited interaction remains significantly underdeveloped. In this letter, we propose an efficient MARL method to address this challenge. Our method enables each vehicle to receive limited messages from surrounding vehicles, which are then used to augment the input representation of the local driving policy. By predicting the next-step state based on the current augmented local state and action, our approach enhances the decision-making capability of each vehicle. Specifically, we design a Self-supervised Message Attention Encoding (SMAE) module that utilizes an attention mechanism to aggregate the received messages and local observations, generating a compact representation. Then, this representation is used in a self-supervised module to predict the next-step state. By jointly training the encoder module and the prediction module, each vehicle effectively leverages the most relevant components of the aggregated representation to improve the learning efficiency of driving policy and alleviate issues related to partial observability in making driving decisions. To validate the effectiveness of our approach, we conduct experiments using an open-source autonomous driving simulator. The simulation results demonstrate that our proposed method outperforms the IPPO, MAPPO and CoPO algorithms in terms of success rate, route completion rate, crash rate, and other relevant metrics.