Expert-Guided Deep Reinforcement Learning for Flexible Job Shop Scheduling Problem

Flexible job shop scheduling (FJSP) is crucial for automated production, ensuring efficiency and flexibility. In recent years, deep reinforcement learning (DRL) has achieved success in solving sequence decision-making problems. However, the efficiency of the generated scheduling plans is often constrained by the dependence of most DRL algorithms on priority dispatching rules (PDR). In order to enable the agent trained in DRL to autonomously choose operations and machines, this paper proposes an expert-guided deep reinforcement learning framework (EGDRL). Based on the representation of scheduling states using a disjunctive graph and an operation-machine topology graph, a graph neural network (GNN) is used to capture the complex relationships between operations and machines. More importantly, in the early stages of training, this paper introduces expert-guided solutions by using PDR to guide the action selection of reinforcement learning, which greatly improves the quality of decision-making. Experimental results consistently show that the proposed method outperforms traditional PDRs and other DRL algorithm. Notably, this superiority is observed across various scales, including larger instances. Additionally, the method exhibits robust performance on instances not encountered during training.