Learning Based Mechanisms for Interference Mitigation in Self-Organized Femtocell Networks

We introduce two mechanisms for interference mitigation, inspired by evolutionary game theory and machine learning to support the coexistence of a macrocell network underlaid with self-organized femtocell networks. In the first approach, stand-alone femtocells choose their strategies, observe the behavior of other players, and make the best decision based on their instantaneous payoff, as well as the average payoff of all other femtocells. We formulate the interactions among selfish femtocells using evolutionary games and demonstrate how the system converges to an equilibrium. In contrast, in the Reinforcement-Learning (RL) approach, information exchange among femtocells is no longer possible and hence each femtocell adapts its strategy and gradually learns by interacting with its environment (i.e., neighboring interferers) through trials-and-errors. Our investigations reveal that through learning, femtocells are able to self-organize by relying only on local information, while mitigating the interference towards the macrocell network. Besides, a trade-off exists where faster convergence is obtained in the evolutionary case as compared to the RL approach, at the expense of more side information. Finally, it is shown that femtocells face an interesting tradeoff of exploration versus exploitation in their learning processes.