Probing an LSTM-PPO-Based reinforcement learning algorithm to solve dynamic job shop scheduling problem

被引:0
|
作者
Chen, Wei [1 ]
Zhang, Zequn [1 ]
Tang, Dunbing [1 ]
Liu, Changchun [1 ]
Gui, Yong [1 ]
Nie, Qingwei [1 ]
Zhao, Zhen [1 ]
机构
[1] Nanjing Univ Aeronaut & Astronaut, Coll Mech & Elect Engn, Nanjing 210016, Peoples R China
基金
中国国家自然科学基金; 中国博士后科学基金;
关键词
Reinforcement learning; Scheduling process; LSTM-PPO; Dynamic job shop scheduling problem;
D O I
10.1016/j.cie.2024.110633
中图分类号
TP39 [计算机的应用];
学科分类号
081203 ; 0835 ;
摘要
With the growth of personalized demand and the continuous improvement in social productivity, the large-scale and few-variety centralized production model is gradually transitioning towards a personalized model of small batches and multiple varieties, which makes the manufacturing process of the job shop increasingly complex. Furthermore, disruptive events such as machinery failures and rush orders in the job shop increase the uncertainty and variability of the production environment. Traditional scheduling methods are usually based on fixed rules and heuristic algorithms, which are difficult to adapt to constantly changing production environments and demands. This may lead to inaccurate scheduling decisions and hinder the optimal allocation of job shop resources. To solve the dynamic job shop scheduling problem (JSP) more effectively, this paper proposes a Reinforcement Learning (RL) optimization algorithm integrating long short-term memory (LSTM) neural network and proximal policy optimization (PPO). It can dynamically adjust scheduling strategies according to the changing production environment, achieving comprehensive status awareness of the job shop environment to make optimal scheduling decisions. First, a state-aware network framework based on LSTM-PPO is proposed to achieve real-time perception of job shop state changes. Then, the state and action space of the job shop are described within the context of the state-aware network framework. Finally, an experimental environment is established to verify the algorithm's effectiveness. Training the LSTM-PPO algorithm makes it feasible to achieve better performance than other scheduling methods. By comparing the initial planning time with the actual completion time of the rescheduling decision under different dynamic disturbances, the efficiency of the proposed algorithm is verified for the dynamic JSP.
引用
收藏
页数:15
相关论文
共 50 条
  • [11] A hybrid evolutionary algorithm to solve the job shop scheduling problem
    Cheng, T. C. E.
    Peng, Bo
    Lu, Zhipeng
    ANNALS OF OPERATIONS RESEARCH, 2016, 242 (02) : 223 - 237
  • [12] Deep reinforcement learning for dynamic distributed job shop scheduling problem with transfers
    Lei, Yong
    Deng, Qianwang
    Liao, Mengqi
    Gao, Shuocheng
    EXPERT SYSTEMS WITH APPLICATIONS, 2024, 251
  • [13] Fuzzy job shop scheduling problem based on deep reinforcement learning
    Zhu, Jia-Zheng
    Zhang, Hong-Li
    Wang, Cong
    Li, Xin-Kai
    Dong, Ying-Chao
    Kongzhi yu Juece/Control and Decision, 2024, 39 (02): : 595 - 603
  • [14] Optimization of job shop scheduling problem based on deep reinforcement learning
    Qiao, Dongping
    Duan, Lvqi
    Li, Honglei
    Xiao, Yanqiu
    EVOLUTIONARY INTELLIGENCE, 2024, 17 (01) : 371 - 383
  • [15] Optimization of job shop scheduling problem based on deep reinforcement learning
    Dongping Qiao
    Lvqi Duan
    HongLei Li
    Yanqiu Xiao
    Evolutionary Intelligence, 2024, 17 : 371 - 383
  • [16] Dynamic flexible job shop scheduling based on deep reinforcement learning
    Yang, Dan
    Shu, Xiantao
    Yu, Zhen
    Lu, Guangtao
    Ji, Songlin
    Wang, Jiabing
    He, Kongde
    PROCEEDINGS OF THE INSTITUTION OF MECHANICAL ENGINEERS PART B-JOURNAL OF ENGINEERING MANUFACTURE, 2024,
  • [17] A self-learning genetic algorithm based on reinforcement learning for flexible job-shop scheduling problem
    Chen, Ronghua
    Yang, Bo
    Li, Shi
    Wang, Shilong
    COMPUTERS & INDUSTRIAL ENGINEERING, 2020, 149
  • [18] An efficient and adaptive design of reinforcement learning environment to solve job shop scheduling problem with soft actor-critic algorithm
    Si, Jinghua
    Li, Xinyu
    Gao, Liang
    Li, Peigen
    INTERNATIONAL JOURNAL OF PRODUCTION RESEARCH, 2024, 62 (23) : 8260 - 8275
  • [19] Knowledge-Based Reinforcement Learning and Estimation of Distribution Algorithm for Flexible Job Shop Scheduling Problem
    Du, Yu
    Li, Jun-qing
    Chen, Xiao-long
    Duan, Pei-yong
    Pan, Quan-ke
    IEEE TRANSACTIONS ON EMERGING TOPICS IN COMPUTATIONAL INTELLIGENCE, 2023, 7 (04): : 1036 - 1050
  • [20] To Solve the Job Shop Scheduling Problem with the Improve Quantum Genetic Algorithm
    Li Dao-wang
    2012 THIRD GLOBAL CONGRESS ON INTELLIGENT SYSTEMS (GCIS 2012), 2012, : 88 - 91