Hierarchical Multi-Agent Deep Reinforcement Learning with an Attention-based Graph Matching Approach for Multi-Domain VNF-FG Embedding

We present a generic multi-agent deep reinforcement learning framework for dynamic multi-domain service provisioning in large-scale networks. We formulate both the assignment of a given sub-VNF-FG to a particular domain and its placement within the assigned local domain as a two-stage graph matching problem. To this purpose, we leverage graph attention networks in combination with hierarchical deep reinforcement learning. The learning process is additionally bootstrapped through self-supervised pre-training for both domain assignment and placement stages. The initial policies are further fine-tuned by the different agents along the learning process to address the evolving states of the domains. We show that our approach provides competitive real-time VNF-FG embedding results while achieving load balancing across the domains without compromising their privacy and autonomy in addition to satisfying QoS constraints. Our intelligent framework also paves the way for novel approaches that can benefit from the inherent graph structure of the problem.