Knowledge-Collaboration-Based Resource Allocation in 6G IoT: A Graph Attention RL Approach

In future 6G-enabled Internet of Things (IoT), users and devices will be divided into numerous distributed domains with smaller base station coverage due to the utilization of terahertz high-frequency band communication. Deep reinforcement learning (DRL) agents will be increasingly deployed in the domain to achieve intelligent service provisioning and resource allocation. However, the existing DRL-based method faces the problem of repeated model training and poor generalization ability when service demand fluctuates and environmental changes occur. In addition, limited training samples in each domain also lead to insufficient model training. Inspired by the collaborative learning of human knowledge, we propose a knowledge collaboration-based resource allocation mechanism for future 6G-enabled IoT and address two basic issues: 1) which agent should collaborate with and 2) how to collaborate. Specifically, we first model the distributed network as a graph and use graph attention (GAT) to capture the fluctuant service demands and time-varying resource capacities in temporal and spatial domains, and then calculate the similarity between the agents. We further propose a collective reinforcement learning (CRL) algorithm that facilitates knowledge collaboration between the agents through the policy distribution. Simulation results verify that the proposed GAT-CRL achieves fast convergence as deep deterministic policy gradient (DDPG) in 4K steps, computing the similarity score more accurately with the increasing attention heads, and achieves higher successful flow than the soft actor-critic (about 3.6%-5.4%) and DDPG (about 14.6%-21%) when adapting to unseen traffic patterns/loads and increasing topology scales.