Enabling Sustainable Underwater IoT Networks With Energy Harvesting: A Decentralized Reinforcement Learning Approach

In this article, we study an energy sustainable Internet-of-Underwater Things (IoUT) network with tidal energy harvesting. Specifically, an analytical model is first developed to analyze the performance of the IoUT network, characterizing the stochastic nature of energy harvesting and traffic demands of IoUT nodes, and the salient features of acoustic communication channels. It is found that the spatial uncertainty resulting from underwater acoustic communication may cause a severe fairness issue. As such, an optimization problem is formulated to maximize the network throughput under fairness constraints, by tuning the random access parameters of each node. Given the global network information, including the number of nodes, energy harvesting rates, communication distances, etc., the optimization problem can be efficiently solved with the Branch and Bound (BnB) method. Considering a realistic network where the network information may not be available at the IoUT nodes, we further propose a multiagent reinforcement learning approach for each node to autonomously adapt the random access parameter based on the interactions with the dynamic network environment. The numerical results show that the proposed learning algorithm greatly improves the throughput performance compared with the existing solutions, and approaches the derived theoretical bound.