Q-routing: From the Algorithm to the Routing Protocol

Routing is a complex task in computer network. This function is mainly devoted to the layer 3 in the Open Standard Interconnection (OSI) model. In the 90s, routing protocols assisted by reinforcement learning were created. To illustrate the performance, most of the literature use centralized algorithms and "home-made" simulators that make difficult (i) the transposition to real networks; (ii) the reproducibility. The goal of this work is to address those 2 points. In this paper, we propose a complete distributed protocol implementation. We deployed the routing algorithm proposed by Boyan and Littman in 1994 based on Q-learning on the network simulator Qualnet. Twenty-five years later, we conclude that a more realistic implementation in more realistic network environment does not give always better Quality of Service than the historical Bellman-Ford protocol. We provide all the materials to conduct reproducible research.