Hybrid Centralized-Distributed Resource Allocation Based on Deep Reinforcement Learning for Cooperative D2D Communications

Device-to-device (D2D) technology enables direct communication between adjacent devices within cellular networks. Due to its high data rate, low latency, and performance improvement in spectrum and energy efficiency, it has been widely investigated and applied as a critical technology in 5G New Radio (NR). Cooperative D2D communication can achieve a win-win situation between cellular users (CUs) and D2D users (DUs) through cooperative relaying techniques. In addition to conventional overlay and underlay D2D communications, it has attracted extensive attention from academic and industrial circles in the past decade. This paper delves into optimizing joint spectrum allocation, power control, and link-matching between multiple CUs and DUs for cooperative D2D communications. Weighted sum energy efficiency (WSEE) is used as the performance metric to address the challenges of green communication and sustainable development. This mixed-integer fractional programming (MIFP) problem can be decomposed into: 1. a classic weighted bipartite graph matching; 2. a series of nonconvex spectrum allocation and power control problems between potentially matched cellular and D2D link pairs. To address this issue, we propose a hybrid centralized-distributed scheme based on deep reinforcement learning (DRL) and the Kuhn-Munkres (KM) algorithm. Leveraging the former, the CUs and DUs autonomously optimize spectrum allocation and power control by only utilizing local information. Then, the base station (BS) determines the link matching utilizing the latter. Simulation results reveal that it achieves more than 96% WSEE of the optimal scheme and 98% WSEE of the centralized DRL-based scheme. It significantly enhances the network convergence speed with low centralized computational overheads. In addition, we also propose and utilize cooperative link sets for corresponding D2D links to accelerate the proposed scheme and reduce signaling exchange further: an average of about 85% WSEE of the optimal scheme is achieved, while more than 50% of signaling and distributed computing overheads are reduced.