Efficient Multi-Objective Optimization on Dynamic Flexible Job Shop Scheduling Using Deep Reinforcement Learning Approach

被引:24
作者
Wu, Zufa [1 ]
Fan, Hongbo [1 ,2 ]
Sun, Yimeng [1 ]
Peng, Manyu [1 ]
机构
[1] Kunming Univ Sci & Technol, Fac Informat Engn & Automat, Kunming 650504, Peoples R China
[2] Kunming Univ Sci & Technol, Fac Modern Agr Engn, Kunming 650500, Peoples R China
关键词
deep reinforcement learning; multi-objective optimization; delay time sum; makespan; dual layer deep Q-network; global optimum; dynamic flexible job shop scheduling;
D O I
10.3390/pr11072018
中图分类号
TQ [化学工业];
学科分类号
0817 ;
摘要
Previous research focuses on approaches of deep reinforcement learning (DRL) to optimize diverse types of the single-objective dynamic flexible job shop scheduling problem (DFJSP), e.g., energy consumption, earliness and tardiness penalty and machine utilization rate, which gain many improvements in terms of objective metrics in comparison with metaheuristic algorithms such as GA (genetic algorithm) and dispatching rules such as MRT (most remaining time first). However, single-objective optimization in the job shop floor cannot satisfy the requirements of modern smart manufacturing systems, and the multiple-objective DFJSP has become mainstream and the core of intelligent workshops. A complex production environment in a real-world factory causes scheduling entities to have sophisticated characteristics, e.g., a job's non-uniform processing time, uncertainty of the operation number and restraint of the due time, avoidance of the single machine's prolonged slack time as well as overweight load, which make a method of the combination of dispatching rules in DRL brought up to adapt to the manufacturing environment at different rescheduling points and accumulate maximum rewards for a global optimum. In our work, we apply the structure of a dual layer DDQN (DLDDQN) to solve the DFJSP in real time with new job arrivals, and two objectives are optimized simultaneously, i.e., the minimization of the delay time sum and makespan. The framework includes two layers (agents): the higher one is named as a goal selector, which utilizes DDQN as a function approximator for selecting one reward form from six proposed ones that embody the two optimization objectives, while the lower one, called an actuator, utilizes DDQN to decide on an optimal rule that has a maximum Q value. The generated benchmark instances trained in our framework converged perfectly, and the comparative experiments validated the superiority and generality of the proposed DLDDQN.
引用
收藏
页数:27
相关论文
共 42 条
[1]   A multi objective optimization approach for flexible job shop scheduling problem under random machine breakdown by evolutionary algorithms [J].
Ahmadi, Ehsan ;
Zandieh, Mostafa ;
Farrokh, Mojtaba ;
Emami, Seyed Mohammad .
COMPUTERS & OPERATIONS RESEARCH, 2016, 73 :56-66
[2]  
Bin Luo, 2021, Journal of Physics: Conference Series, V1848, DOI 10.1088/1742-6596/1848/1/012029
[3]   Hierarchical Reinforcement Learning for Multi-Objective Real-Time Flexible Scheduling in a Smart Shop Floor [J].
Chang, Jingru ;
Yu, Dong ;
Zhou, Zheng ;
He, Wuwei ;
Zhang, Lipeng .
MACHINES, 2022, 10 (12)
[4]   Deep Reinforcement Learning for Dynamic Flexible Job Shop Scheduling with Random Job Arrival [J].
Chang, Jingru ;
Yu, Dong ;
Hu, Yi ;
He, Wuwei ;
Yu, Haoyu .
PROCESSES, 2022, 10 (04)
[5]   APPROXIMATE SOLUTIONS OF THE BELLMAN EQUATION OF DETERMINISTIC CONTROL-THEORY [J].
DOLCETTA, IC ;
ISHII, H .
APPLIED MATHEMATICS AND OPTIMIZATION, 1984, 11 (02) :161-181
[6]  
Fan J., 2020, P 2 C LEARNING DYNAM
[7]   Effective and interpretable dispatching rules for dynamic job shops via guided empirical learning [J].
Ferreira, Cristiane ;
Figueira, Goncalo ;
Amorim, Pedro .
OMEGA-INTERNATIONAL JOURNAL OF MANAGEMENT SCIENCE, 2022, 111
[8]  
Garey M. R., 1976, Mathematics of Operations Research, V1, P117, DOI 10.1287/moor.1.2.117
[9]   Research on Adaptive Job Shop Scheduling Problems Based on Dueling Double DQN [J].
Han, Bao-An ;
Yang, Jian-Jun .
IEEE ACCESS, 2020, 8 :186474-186495
[10]   Deep reinforcement learning based AGVs real-time scheduling with mixed rule for flexible shop floor in industry 4.0 [J].
Hu, Hao ;
Jia, Xiaoliang ;
He, Qixuan ;
Fu, Shifeng ;
Liu, Kuo .
COMPUTERS & INDUSTRIAL ENGINEERING, 2020, 149