Communication-Efficient Multi-Agent Actor-Critic Framework For Distributed Optimization Of Resource Allocation in V2X Networks

The vehicular communication technology has enabled new services for drivers and passengers with different Quality of Service (QoS) demands. Due to network resource limitation, a cooperative resource allocation scheme between Vehicle to Infrastructure (V2I) links and Vehicle to Vehicle (V2V) links is needed. In literature, the resource allocation problem can be solved using a cooperative multi-agent reinforcement learning (MARL) framework. Such framework can be implemented in a central controller that receives observations and rewards of all agents, and then calculates the action for each agent accordingly. However, such a central controller may not be realistic for a vehicular network which is highly flexible and requires realtime decision making. Therefore, decentralized schemes where the agents exchange messages to maximize their average rewards would be more appropriate. Nevertheless, decentralized training increases communication costs among the agents, which is a challenging issue for a vehicular network with limited communication bandwidth. This paper proposes an Attentional Double Hierarchical Advantage Actor-Critic (ADHA2C) to address this issue. Specifically, ADHA2C adopts an attention mechanism added to the actor part to classify the important messages sent from other agents in the network. Our extensive experiments and analysis show that the proposed method approximates the performance of the upper bound model, and disturbance in the learning phase can be avoided through our proposed attention mechanism.