Relay Management in Energy-aware Cooperative Ad Hoc Networks: A Perspective of Stochastic Game

Energy efficiency is a crucial requirement for energy-aware ad hoc networks. The efficient relay management, i.e., the joint issue of relay node selection and transmit power control, can be applied to reduce and balance energy consumption. In most existing works, however, the nodes are required to share the distributed information, e. g., the channel knowledge, the updates on relay node selection and power strategies, which incurs the frequent exchange of information. The requirement of significant communication overheads among nodes not only impede fully distributed solutions, but also disagrees with the energy-saving goal. In this work, we formulate the energy-efficient relay management problem as a discrete, stochastic game for a multi-source multi-relay ad hoc network. In the proposed game, each potential relay node is viewed as a player to search for the best action in the probability space with the incomplete distributed information. We investigate the achievable performance of the proposed game in terms of the existence of Nash equilibrium, its expression by using the support and programming methods, and its Pareto optimality. Moreover, we propose a low-complexity, distributed learning algorithm based on the linear reward-inaction procedure. The properties of convergence and learning rate of the algorithm are analyzed.