Deep Q-Network-Based Intelligent Routing Protocol for Underwater Acoustic Sensor Network

This article proposes a deep Q-network (DQN)-based intelligent routing (DQIR) protocol for the underwater acoustic sensor networks (UASNs). The routing decision problem is modeled as a Markov decision process (MDP). The DQN is applied to solve the MDP, in which the agent is trained to select the forwarder with the highest reward as the next hop. The optimal policy for the agent is to choose a routing that balances the residual energy of different nodes while minimizing the routing distance, thereby improving the network lifetime and decreasing the average time delay. To evaluate its performance, we developed the proposed algorithm on an Aqua-Sim Next Generation (Aqua-Sim NG) platform and using the artificial intelligence (AI) framework. According to the simulation results, DQIR consumes less energy than both the depth-based routing (DBR) protocol and DQN-based energy and latency-aware routing (DQELR). Furthermore, compared with DBR and DQELR, DQIR increases the network lifetime and reduces the average time delay.