A Deep Reinforcement Advantage Actor-Critic-Based Co-Evolution Algorithm for Energy-Aware Distributed Heterogeneous Flexible Job Shop Scheduling

With the rapid advancement of the manufacturing industry and the widespread implementation of intelligent manufacturing systems, the energy-aware distributed heterogeneous flexible job shop scheduling problem (DHFJSP) has emerged as a critical challenge in optimizing modern production systems. This study introduces an innovative method to reduce both the makespan and the total energy consumption (TEC) in the context of the DHFJSP. A deep reinforcement advantage Actor-Critic-based co-evolution algorithm (DRAACCE) is proposed to address the issue, which leverages the powerful decision-making and perception abilities of the advantage Actor-Critic (AAC) method. The DRAACCE algorithm consists of three main components: First, to ensure a balance between global and local search capabilities, we propose a new co-evolutionary strategy. This enables the algorithm to explore the solution space efficiently while maintaining robust exploration and exploitation. Next, a novel evolution strategy is introduced to improve the algorithm's convergence rate and solution diversity, ensuring that the search process is both fast and effective. Finally, we integrate deep reinforcement learning with the advantage Actor-Critic framework to select elite solutions, enhancing the optimization process and leading to superior performance in minimizing both TEC and makespan. Extensive experiments validate the effectiveness of the proposed DRAACCE algorithm. The experimental results show that DRAACCE significantly outperforms existing state-of-the-art methods on all 20 instances and a real-world case, achieving better solutions in terms of both makespan and TEC.