Scaling Collaborative Space Networks with Deep Multi-Agent Reinforcement Learning

Future space communication architectures deployed across heterogeneous space systems will require novel methods of coordinating inter-system communication and command distribution. As network complexity increases in time and distance, the ability to facilitate command and control across a large number of systems is a significant constraint on mission performance. This study presents the application of multi-agent reinforcement learning (MARL) to demonstrate a collaborative mesh network of inter-satellite links that self-configure and self-optimize in response to varying mission data needs. This paper explores methods of scaling distributed reinforcement learning-based approaches where satellites modeled as RL agents can observe their local wireless environment, share knowledge with other satellites, and cooperatively achieve network-wide mission objectives. It also implements a transfer learning approach for increasing the network size of a distributed, multi-agent system without modifying action and observation spaces.