Reinforcement learning-based resource allocation for dynamic aggregated WiFi/VLC HetNet

High transmission rates and low power consumption make visible light communication (VLC) a highly promising supplementary technology for the next generation of mobile communication. Taking into account the limited coverage area, VLC can be combined with WiFi as a heterogeneous network (HetNet), thanks to their non-overlapping spectrum. In order to fully utilize the advantages of the WiFi and VLC technologies, a new aggregated WiFi/VLC HetNet consisting of a single WiFi AP and multiple VLC APs, is designed, where the user equipment (UE), for the first time, can access multiple VLC APs and one WiFi AP simultaneously, and be allowed to acquire multiple resource blocks (RB) in each AP at the same time. To optimize the performance of the above designed HetNet, a multi-objective optimization problem (MOOP) is formulated, which aims to maximize system throughput while reducing the handover rate. Since the above MOOP is nonconvex and nonlinear, the traditional resource allocation (RA) algorithm has a complex calculation process and poor timeliness performance to deal with this problem. To solve the above MOOP, a reinforcement learning (RL)-based RA algorithm is proposed, and considering the RB handover overhead in the aggregated WiFi/VLC HetNet, a reward function related to the system throughput and the UE handover rate is carefully designed. System throughput, system handover rate, user satisfaction, as well as user fairness performance are analyzed under three typical indoor illumination layouts. Finally, compared with the greedy algorithm and the hypergraph-based carrier aggregation algorithm, numerical results show that the proposed RL-based RA algorithm could improve the system throughput over the former two algorithms by 30.26% and 19.71%, while reducing the system handover rate by 0.15% and 0.02%, respectively.