Deep Reinforcement Learning-Based Optimization Method for D2D Communication Energy Efficiency in Heterogeneous Cellular Networks

Given the context of the challenge of exponentially increasing data traffic on communication networks brought by the 5G era, this paper focuses on how to apply deep reinforcement learning (DRL) techniques to solve the problem of optimizing the energy efficiency of D2D communications in heterogeneous cellular network environments. We propose a joint resource allocation scheme based on multi-intelligent deep reinforcement learning, which enables D2D devices to intelligently switch between license-free and optimal license bands and adjust the transmit power in real time to maximize the energy efficiency improvement. In this work, a multi-intelligent deep reinforcement learning framework is designed to enable D2D users in heterogeneous networks to make collaborative decisions and dynamically adjust their communication strategies according to real-time network status and environmental changes. In this paper, a deep Q-network model with a graph attention network (GAT) as the core structure is constructed; this model can cope with the complexity and diversity of network states and learn and execute optimal resource allocation strategies. In this paper, we propose a targeted loss function design that balances the optimization goal of D2D communication energy efficiency with network stability and long-term gains. Through rigorous simulation experiments, this paper verifies that a DRL-based approach can significantly improve the energy efficiency of D2D communications in heterogeneous cellular networks in real-world scenarios while ensuring the stability of the quality of service (in terms of, e.g., rate, delay, and resource utilization).