Subcarrier power control for URLLC communication system via multi-agent deep reinforcement learning in IoT network

Designing an intelligent resource allocation scheme to achieve the performance requirements of internet of things (IoT) devices for the future ultra-reliable low-latency communication (URLLC) network is a challenging task. In this paper, we formulate a joint blocklength allocation and power control optimisation problem to maximise the sum-rate performance with the short data packet in an uplink URLLC communication system. To alleviate this non-convex optimisation problem under the subcarrier power, blocklength and rate constraints, we firstly transfer it into a multi-agent reinforcement learning (RL) problem, in which each subcarrier works as the agent to decide its own power intelligently. Then a distributed blocklength allocation and power control scheme is proposed based on deep Q-network (DQN). To improve the rate performance in the dynamic communication environment, we design the segmented reward function depending on the communication rate and blocklength under different conditions, and adopt the experience replay strategy to avoid the dependency of training data. Finally, the simulation results show that the proposed scheme achieve the effectiveness and convergence under different settings compared to benchmark schemes.