Deep reinforcement learning-based joint optimization of computation offloading and resource allocation in F-RAN

The fog radio access network (F-RAN) has been regarded as a promising wireless access network architecture in the fifth generation (5G) and beyond systems to satisfy the increasing requirements for low-latency and high-throughput services by providing fog computing. However, because the cloud computing centre and fog computing-enabled access points (F-APs) in the F-RAN have different computation and communication capabilities, it is crucial to make an efficient computation offloading and resource allocation strategy that can fully exploit the potential of the F-RAN system. In this paper, the authors investigate a decentralized low-complexity deep reinforcement learning (DRL)-based framework for joint computation task offloading and resource allocation in the F-RAN, which supports assistive computing-enabled tasks offloading between F-APs. Considering the constraints of task latency, wireless transmission rate, transmission power, and computational resource capacity, the authors formulate the system processing efficiency maximization problem by jointly optimizing offloading mode selection, channel allocation, power control, and computation resource allocation in the F-RAN. To solve this non-linear and non-convex problem, the authors propose a federated DRL-based computation offloading and resource allocation algorithm to improve the task processing efficiency and ensure privacy in the system, which can significantly reduce the computing complexity and signalling overhead of the training process compared with the centralized learning-based method. Specifically, each local F-AP agent consists of dueling deep Q-network (DDQN) and deep deterministic policy gradient (DDPG) networks to appropriately deal with discrete and continuous valuable action spaces, respectively. Finally, the simulation results show that the proposed federated DRL algorithm can achieve significant performance improvements in terms of system processing efficiency and task latency compared with other benchmarks.