QLFR: A Q-Learning-Based Localization-Free Routing Protocol for Underwater Sensor Networks

Designing a routing protocol for underwater sensor networks is a great challenge due to characteristics of high energy consumption and high latency. This paper investigates a Q-learning-based localization-free routing protocol (QLFR) to prolong the lifetime as well as reduce the end-to-end delay for underwater sensor networks. Aiming to seek optimal routing policies, Q-value is calculated by jointly considering residual energy and depth information of sensor nodes throughout the routing process. More specifically, we define two cost functions (depth-related cost and energy-related cost) for Q-learning, in order to reduce delay and extend the network lifetime. In addition, a holding time mechanism for packet forwarding is designed according to the priority of forwarding nodes. The key contribution lies in: 1) a novel Q-learning-based routing protocol for UWSNs; 2) a new holding time mechanism for packet forwarding; and 3) a packet-delivery-ratio-based scheme to further reduce unnecessary transmissions. Extensive simulation results demonstrate superiority performance of our routing protocol in terms of reducing end-to-end delay and extending the network lifetime.