MAC Protocol for Underwater Acoustic Multi-Cluster Networks Based on Multi-Agent Reinforcement Learning

Underwater acoustic multi-cluster networks (UWA-MCNs) have gained prominence due to their expansive coverage and ease of deployment. A critical challenge in the design of medium access control (MAC) protocols for UWA-MCNs stems from the spatio-temporal uncertainty caused by significant propagation delays. To address this challenge, we propose a novel distributed MAC protocol named delay-assisted multi-cluster access (DA-MCA), leveraging the delay-assisted multi-agent deep reinforcement (DAMA) algorithm. We introduce a mechanism termed historical information assistance, enabling nodes to make autonomous decisions in conditions of partial observation. Moreover, we establish the intelligent experience splicing mechanism, tailored to handle reward shuffling resulting from varying delay among nodes. DAMA is employed to learn an optimal strategy for interaction with the environment, ensuring minimal communication overhead, high utilization, and collision avoidance at the access point. DA-MCA overcomes the uncertainty challenge, maximizing UWA-MCN throughput. Through comprehensive experiments, we validate our approach's effectiveness, demonstrating that DA-MCA significantly enhances total throughput compared to baseline methods.