An efficient routing access method based on multi-agent reinforcement learning in UWSNs

A large proportion of underwater data is collected in deep sea. Compared with the direct bottom-to-surface acoustic links, underwater sensor networks (UWSNs) with hierarchical network model topology are more efficient at transmitting huge amounts of data to sea surface. Base on reinforcement learning, an adaptive modulation and coding in depth based router (MC-DBR) algorithm was proposed. The MC-DBR is designed to reduce the energy consumption, time delay etc., while improve the communication performance. In MC-DBR, each node firstly uses HELLO packets to sense the neighbouring channel states. Then, each node updates its Q-value by multi-agent reinforcement learning based modulation and coding method (MARL-MC) algorithm. The energy consumption, the time delay, the modulation and coding methods and the packets collisions etc. are considered in MARL-MC to improve the overall performance of the whole network. The convergence and computation complexity of the MC-DBR were analyzed in detail. The performance of the MC-DBR was compared with the benchmark algorithms. The results showed that the MC-DBR can obtain lower end-to-end delay, higher packet delivery rate and lower average remaining energy of the network.