Reinforcement Learning Based QoS-Provisioning Over Energy-Harvesting 5G Wireless Ad-Hoc Networks

To relieve the energy scarcity problem introduced by the explosively growing number of wireless multimedia services, researchers have proposed and developed the energy harvesting (EH) technique over 5G wireless ad-hoc networks where the mobile devices can harvest energy from various external sources. Moreover, statistical delay-bounded quality-of-service (QoS) provisioning has been developed and extensively studied for guaranteeing wireless multimedia data transmissions over 5G ad-hoc networks. However, because of the stochastic nature of EH and data arrival processes, it is challenging to design an optimal EH policy as well as resource allocation policies to guarantee statistical delay-bounded QoS requirements. Along this direction, the reinforcement learning algorithms have been designed to select optimal EH and resource allocation policies by allowing the mobile users to learn from the different network states and historical behavior until an optimal response set is reached. By leveraging the benefits of reinforcement learning, in this paper we propose a reinforcement learning based algorithm for designing optimal EH and resource allocation policies while satisfying heterogeneous statistical delay-bounded QoS constraints over 5G wireless ad-hoc networks. In particular, we first establish system models for EH and wireless data transmissions. Then, we formulate the effective-capacity maximization problem to find optimal power-subchannel allocation policies over EH based 5G wireless ad-hoc networks. To solve such a maximization problem, we apply a reinforcement learning based EH algorithm for deriving an optimal EH policy and a corresponding optimal resource allocation policies. We also conduct a set of performance evaluations to show that our proposed reinforcement learning based EH scheme outperforms existing schemes subject to the heterogeneous statistical delay-bounded QoS constraints over 5G wireless ad-hoc networks.