Graph Q-learning Assisted Ant Colony Optimization for Vehicle Routing Problems with Time Windows

Vehicle routing problem with time windows (VRPTW) is a typical class of constrained path planning problems in the field of combinatorial optimization. VRPTW considers a delivery task for a given set of customers with time windows, and the target is to find optimal routes for a group of vehicles that can minimize the total transportation cost. The traditional heuristics suffer from several limitations when solving VRPTW, such as poor scalability, sensitivity to hyperparameters and difficulty in handling complex constraints. Recent advance in machine learning makes it possible to enhance heuristic approaches via learned knowledge. In this paper, we propose a graph Q-learning assisted ant colony optimization algorithm named GQL-ACO to solve VRPTW. Compared to vanilla ant colony optimization (ACO), our proposed method first employs the learned heuristic values by using graph Q learning, instead of handcrafted ones, to define the hyperparameters of ACO. Second, we design a collaborative search strategy by combining ACO and Q-learning effectively, which can adaptively adjust the hyperparameters of ACO based on the search experiences.