Bargaining solutions in heterogeneous networks: A reinforcement learning-based approach

To enhance the performance and the coverage area of the next-generation heterogeneous wireless networks (HetNets), smaller cells such as femtocells are deployed. A reasonable resource allocation strategy is crucial for the operation of such systems. In this work, the Nash bargaining solution (NBS), Kalai-Smorodinsky bargaining solution (KSBS), and a proposed bargaining solution, which is the combination of NBS and KSBS, are studied to ensure the efficiency and fairness in the transmission rate allocation among femtocells. The mentioned bargaining solutions are optimised subject to the rate constraints due to the Han-Kobayashi (HK) rate splitting scheme, and a maximum tolerable interference caused by femtocell base stations (FBSs) at the macrocell user equipment. Two methods are used to address the bargaining solutions' optimisations. First, the conventional optimisation-based method is used for the situations where the channel gains are known, then a reinforcement learning-based algorithm for a real environment is proposed without being aware of the channel gains. To accelerate the speed of the learning process and to initialise the values of quality functions, the hotbooting technique is used. The bargaining solutions are compared by fairness and efficiency criteria in both methods. Numerical simulations show that the proposed reinforcement learning-based algorithm achieves the same performance derived from the conventional optimisation method. First, this paper studies the two well-known Nash and Kalai-Smorodinsky bargaining solutions as the good tools to establish fairness in rate allocation problem among femtocells in a HetNet. Then a new bargaining solution is proposed as the combination of the two bargaining solutions. The fairness and the efficiency of the allocated rates to femtocells as the output of optimisation problems of bargaining solutions are compared via Jain index criterion (as the fairness metric) and inverse price of anarchy criterion (as the efficiency metric). Moreover, the femtocell base stations are allowed to use Han-Kobayashi signalling scheme to send their signals in private and common messages which results in the best-known achievable rate region. Two methods are used to solve the optimisation problems: the conventional optimisation method and the reinforcement learning-based method (to formulate a real-world model with the unknown channel gains). It is shown that the proposed bargaining solution (in both methods) results in fairer rates than Nash and Kalai-Smorodinsky bargaining solutions while all of them are efficient.