Network Function Placement in Virtualized Radio Access Network with Reinforcement Learning Based on Graph Neural Network

In 5G and beyond 5G networks, function placement is a crucial strategy for enhancing the flexibility and efficiency of the Radio Access Network (RAN). However, demonstrating optimal function splitting and placement to meet diverse user demands remains a significant challenge. The function placement problem is known to be NP-hard, and previous studies have attempted to address it using Deep Reinforcement Learning (DRL) approaches. Nevertheless, many existing methods fail to capture the network state in RANs with specific topologies, leading to suboptimal decision-making and resource allocation. In this paper, we propose a method referred to as GDRL, which is a deep reinforcement learning approach that utilizes graph neural networks to address the functional placement problem. To ensure policy stability, we design a policy gradient algorithm called Graph Proximal Policy Optimization (GPPO), which integrates GNNs into both the actor and critic networks. By incorporating both node and edge features, the GDRL enhances feature extraction from the RAN's nodes and links, providing richer observational data for decision-making and evaluation. This, in turn, enables more accurate and effective decision outcomes. In addition, we formulate the problem as a mixed-integer nonlinear programming model aimed at minimizing the number of active computational nodes while maximizing the centralization level of the virtualized RAN (vRAN). We evaluate the GDRL across different RAN scenarios with varying node configurations. The results demonstrate that our approach achieves superior network centralization and outperforms several existing methods in overall performance.