A Transmission-Reliable Topology Control Framework Based on Deep Reinforcement Learning for UWSNs

This article focuses on the topology optimization problem to decrease transmission delay and prolong network lifetime on the premise of reliable transmission in underwater wireless sensor networks (UWSNs). This is extremely challenging owing to dynamic ocean current movement, harsh underwater communication channel, and increasingly demanding application requirements. With the development of software-defined network architectures, the centralized topology control (TC) strategy with a global perspective in UWSNs is expected to become a more effective way to tackle the above challenges compared with the existing distributed and heuristic TC strategies involving local network state information. Therefore, we first transform the topology optimization problem of UWSNs into an integer nonlinear programming (INLP) model and design a centralized TC framework to solve the INLP model. In this framework, a TC center is built to periodically generate the network topology for UWSNs according to the current network state information. Further, an efficient topology generation algorithm based on deep reinforcement learning (TGA-DRL) is proposed in the TC center. In TGA-DRL, to reduce computing overhead and improve operational efficiency, we formulate an action-space narrowed Markov decision process suitable for network topology generation and solve it with the aid of the rainbow algorithm which is a deep reinforcement learning model. Finally, the performance of our centralized TC framework is verified in terms of the node out-degree, algorithm convergence, and optimization effect.