Throughput Optimization for Grant-Free Multiple Access With Multiagent Deep Reinforcement Learning

Grant-free multiple access (GFMA) is a promising paradigm to efficiently support uplink access of Internet of Things (IoT) devices. In this paper, we propose a deep reinforcement learning (DRL)-based pilot sequence selection scheme for GFMA systems to mitigate potential pilot sequence collisions. We formulate a pilot sequence selection problem for aggregate throughput maximization in GFMA systems with specific throughput constraints as a Markov decision process (MDP). By exploiting multiagent DRL, we train deep neural networks (DNNs) to learn near-optimal pilot sequence selection policies from the transition history of the underlying MDP without requiring information exchange between the users. While the training process takes advantage of global information, we leverage the technique of factorization to ensure that the policies learned by the DNNs can be executed in a distributed manner. Simulation results show that the proposed scheme can achieve an average aggregate throughput that is within 85% of the optimum, and is 31%, 128%, and 162% higher than that of acknowledgement-based GFMA, dynamic access class barring, and random selection GFMA, respectively. Our results also demonstrate the capability of the proposed scheme to support IoT devices with specific throughput requirements.