A multi-objective genetic algorithm based on two-stage reinforcement learning for green flexible shop scheduling problem considering machine speed

The consumption of energy and resources in the manufacturing industry has garnered significant attention due to the increasingly severe environmental issues. Green shop scheduling research is focused on optimizing economic and environmental indicators within the current manufacturing model. This paper specifically addresses the flexibility of job-shop scheduling problem considering machine speed (CMS-FJSP), as different machine speeds during the production process can impact energy and resource consumption. The objectives of this study include minimizing the makespan, total energy consumption, and tool wear. To tackle this problem, a multi-objective genetic algorithm that incorporates a two-stage reinforcement learning approach is proposed. In light of the characteristics of the problem, a three-layer encoding approach is suggested, which encompasses machine allocation, operation sequencing, and machine speed selection. Additionally, a decoding method that integrates energy-saving strategies is proposed to enhance the optimization process. To improve the quality of the population, three distinct initialization methods have been developed. Furthermore, a parameter adjustment strategy informed by two-stage reinforcement learning is introduced. This strategy incorporates a state set and action set tailored to the unique characteristics of two-stage reinforcement learning, alongside corresponding reward mechanisms. In 30 test cases, the proposed algorithm demonstrates superior uniformity and convergence compared to five classical algorithms. In a practical case within a hydraulic component production workshop conducted at a hydraulic component company, the proposed algorithm generates 26 scheduling schemes with different focuses, achieving a 14.39% reduction in makespan, a 2.13% decrease in energy consumption, and a 10.65% reduction in tool wear.