Deep Q-Learning for Routing Schemes in SDN-Based Data Center Networks

In order to adapt to the rapid development of cloud computing, big data, and other technologies, the combination of data center networks and SDN is proposed to make network management more convenient and flexible. With this advantage, routing strategies have been extensively studied by researchers. However, the strategies in the controller mainly rely on manual design, the optimal solutions are difficult to be obtained in the dynamic network environment. So the strategies based on artificial intelligence (AI) are being considered. This paper proposes a novel routing strategy based on deep Q-learning (DQL) to generate optimal routing paths autonomously for SDN-based data center networks. To satisfy the different demands of mice-flows and elephant-flows in data center networks, deep Q networks are trained for them respectively to achieve low latency and low packet loss rate for mice-flows as well as high throughput and low packet loss rate for elephant-flows. Furthermore, with the consideration of the distribution of traffic and the limitated resources of data center networks and SDN, we choose port rate and flow table utilization to describe the network state. Simulation results show that compared with Equal-Cost Multipath (ECMP) routing and Selective Randomized Load Balancing (SRL)+FlowFit, the proposed routing scheme can reduce both the average delay of mice-flows and average packet loss rate, while increase the average throughput of elephant-flows.