Optimization of Energy Efficiency for Uplink mURLLC Over Multiple Cells Using Cooperative Multiagent Reinforcement Learning

Multiagent reinforcement learning (RL) has recently been adopted to solve massive ultrareliable and low-latency communications (mURLLC) energy efficiency (EE) optimization problem in a single-cell cellular network under random access. Bursty traffic is an important characteristic of mURLLC users (UEs). This characteristic and its impact on the RL scheme are generally ignored in many RL-based studies related to the optimization of EE for uplink mURLLC. Moreover, in a smart factory with multiple cells, intercell interference and shadow fading further complicate EE optimization. To address these issues, we propose a novel cooperative multiagent scheme to maximize the long-term EE in a multicell cellular network with mURLLC bursty traffic and a K-repetition scheme by optimizing the repetition value and transmission power. A UE clustering algorithm and an intermittent learning mode are adopted to reduce the computational complexity and mitigate the impact of bursty traffic on the RL scheme. A proper reward function is designed to address both long-term EE maximization and the number of successfully served UEs under high-reliability requirement. The simulation results show that our proposed cooperative multiagent reinforcement learning scheme greatly outperforms other existing schemes in terms of long-term accumulated EE and the number of successfully served UEs.