Joint Optimization of BS Clustering and Power Control for NOMA-Enabled CoMP Transmission in Dense Cellular Networks

Non-orthogonal multiple access (NOMA)-enabled coordinated multipoint (CoMP) transmission has great potential in balancing spectrum utilization and interference mitigation in dense cellular networks. However, NOMA reforms the spectrum sharing policy of CoMP transmission due to introducing additional interference among coordinated base stations (BSs), which deteriorates the CoMP transmission rate. In this paper, we investigate the joint optimization of BS clustering and power control for NOMA-enabled CoMP transmission in dense cellular networks to maximize system sum-rate. We first characterize the impact of interference among coordinated BSs on CoMP transmission rate and find that the BS clustering is dependent on NOMA user grouping and restricted by the NOMA decoding condition. We then derive a tight lower bound on the system sum-rate and exploit it to design a joint BS clustering and power control scheme. Specifically, a power control algorithm is designed by a penalty convex-concave procedure to satisfy the NOMA decoding condition and users' rate requirements. A BS clustering algorithm based on successive convex approximation is designed to iteratively update the BS clustering and NOMA user grouping to increase system sum-rate. Finally, two algorithms are alternately performed until all users successfully access and the system sum-rate converges. Simulation results show that the proposed scheme can efficiently alleviate interference among coordinated BSs to improve system sum-rate and spectrum efficiency of NOMA-enabled CoMP transmission even under high user density.