Hierarchical Adaptation of Multiagent Deep Reinforcement Learning for Multi-Domain Uncrewed Aerial and Ground Vehicle Coordination

The coordination of Uncrewed Aerial Vehicles (UAVs) and Uncrewed Ground Vehicles (UGVs) enhances several commercial and military applications, yet it confronts specific challenges, including restricted vertical access, limited operational range, and power constraints, alongside the necessity of maintaining robust communication links between vehicles. This work introduces a hierarchical adaptation of a multi-agent deep reinforcement learning (MADRL) model, termed Fusion Multi-Actor-Attention-Critic (F-MAAC), engineered for the adept management of UAVs and UGVs in intricate settings. The proposed model combines a high-level partner selector with a low-level action selector, employing temporal abstraction and sophisticated communication protocols to amplify operational efficacy and precision in task execution. Notably, the integration enables a Lead UAV Agent to efficiently coordinate the surveillance of environmental variables that are beyond the perceptual field of the UGVs. Comparative analysis between our proposed model and other models underscores the superior performance and enhanced coordination facilitated by the hierarchical F-MAAC model, advocating its utility in complex operational contexts.