Deep Reinforcement Learning for Network Slice Placement and the DeepNetSlice Toolkit

Reinforcement Learning (RL) is gaining increasing interest in the development of solutions for the placement of network slices on top of a physical network infrastructure. Although the majority of related works exploits simulations for training and evaluation purposes, authors typically have their own definition of the problem at hand. This leads to significant implementation efforts, including the development of the environment. To address the lack of RL platforms tailored to the problem of Network Slice Placement (NSP), we propose DeepNetSlice, a highly customizable and modular toolkit, serving as a RL environment and strongly integrated with main RL libraries. We validate our toolkit through comparative evaluations with related work and the development of a RL approach for the NSP problem that improves state-of-the-art results by up to 8.95% in acceptance ratio. Our RL approach exploits a graph convolutional network and leverages a multi-worker advantage actorcritic learning algorithm with generalized advantage estimation. Two further contributions of this work are the application of invalid action masking in the context of NSP, and a novel analysis on the generalization capabilities of our method, where we show its effectiveness with dynamic network infrastructures, unlike previous works in literature.