QTAR: A Q-learning-based topology-aware routing protocol for underwater wireless sensor networks*

In this paper, an energy-efficient Q-learning-based routing protocol, called the Q-learning-based topology-aware routing (QTAR) protocol, is proposed for underwater wireless sensor networks. Unlike existing protocols, QTAR considers the network topology to determine the next-forwarder (NF) candidates along the routing path and adopts Q-learning to aid in the optimal global decision-making of an NF from the NF candidates. In addition, QTAR utilizes implicit cut-vertex recognition to optimize NF selection, alleviating the energy wastage that arises from forwarding data packets away from the sink. In our study, we evaluated the performance of QTAR by comparing it with the Q-learning-based energy-efficient and lifetime-aware routing protocol (QELAR), energy-efficient depth-based routing protocol (EEDBR), Q-learning-based delay-aware routing (QDAR), and reinforcement learning-based opportunistic routing protocol (RLOR) in terms of the energy consumption, latency, and network lifetime. Our results revealed that QTAR demonstrated the advantages of a lower energy consumption, shorter latency, and longer network lifetime in the percentage ranges of 26.08 to 70.12, 22.2 to 50, and 37.8 to 75, respectively, than QELAR, EEDBR, QDAR, and RLOR.