Communication-Efficient Multi-Robot Exploration Using Coverage-Biased Distributed Q-Learning

Frontier exploration and reinforcement learning have historically been used to solve the problem of enabling many mobile robots to autonomously and cooperatively explore complex surroundings. These methods need to keep an internal global map for navigation, but they do not consider the high costs of communication and information sharing between robots. This study offers CQLite, a novel distributed Q-learning technique that minimizes data communication between robots while achieving rapid convergence and thorough coverage in multi-robot exploration. The proposed CQLite method uses ad hoc map merging, and selectively shares updated Q-values at recently identified frontiers to reduce communication costs significantly. The theoretical analysis of CQLite's convergence and efficiency and extensive numerical verification on simulated indoor maps utilizing several robots demonstrate the method's novelty. With over 2x reductions in computation and communication alongside improved mapping performance, CQLite outperformed cutting-edge multi-robot exploration techniques like Rapidly-exploring Random Trees and Deep Reinforcement Learning.