A multi-level action coupling reinforcement learning approach for online two-stage flexible assembly flow shop scheduling

被引:0
作者
Qiu, Junhao [1 ]
Liu, Jianjun [1 ]
Li, Zhantao [1 ]
Lai, Xinjun [1 ]
机构
[1] Guangdong Univ Technol, Guangdong Prov Key Lab Comp Integrated Mfg Syst, State Key Lab Precis Elect Mfg Technol & Equipment, Guangzhou 510006, Guangdong, Peoples R China
关键词
Flexible assembly flow shop; Multi-product delivery; Online scheduling; Reinforcement learning; Multi-level rule combinations; Asynchronous execution; BOUND ALGORITHM; SETUP TIMES; MINIMIZE; SYSTEM;
D O I
暂无
中图分类号
T [工业技术];
学科分类号
08 ;
摘要
Multi-product centralized delivery and kitting assembly present significant challenges to hierarchical co- processing in multi-stage manufacturing systems. The combinations of priority dispatching rules at each level are transiently adaptive, and the performance in online scheduling deteriorates rapidly with changing environment. This paper investigates the selection of rule combinations for sustained high-performance responsive scheduling in two-stage flexible assembly flow shop scheduling problem with asynchronous execution and complex decision correlation. A Multi-Level Action Coupling Deep Q-Network (MALC-DQN) approach is proposed for adaptive integrated scheduling in hybrid processing and assembly shops. Firstly, the problem is skillfully established as an event-triggered integrated decision markov decision process. The prioritized batch experience replay mechanism is employed to retain the complete correlation information of key decision sequences. Then, coupling and sequence feature extraction modules are developed to enhance the agent's ability to perceive execution process and the environment. Furthermore, the multi-level wait- limit mechanism and efficient action filtering mechanism are designed to mitigate ineffective waiting waste and action space explosion during learning. Finally, a series of sophisticated experiments are conducted to validate the effectiveness of the proposed methodology. In 20 actual instances of different sizes, MLAC-DQN outperformed its closest competitor, with a 26.6% improvement in average tardiness. Moreover, extraordinary robustness is demonstrated in 16 sets of experiments involving different configurations of resources, orders, and arrival concentration levels.
引用
收藏
页码:370 / 370
页数:1
相关论文
共 45 条
[1]   A data-driven scheduling approach to smart manufacturing [J].
Alejandro Rossit, Daniel ;
Tohme, Fernando ;
Frutos, Mariano .
JOURNAL OF INDUSTRIAL INFORMATION INTEGRATION, 2019, 15 :69-79
[2]   A survey of scheduling problems with setup times or costs [J].
Allahverdi, Ali ;
Ng, C. T. ;
Cheng, T. C. E. ;
Kovalyov, Mikhail Y. .
EUROPEAN JOURNAL OF OPERATIONAL RESEARCH, 2008, 187 (03) :985-1032
[3]   Minimizing the number of tardy jobs on a two-stage assembly flowshop [J].
Allahverdi, Ali ;
Aydilek, Asiye ;
Aydilek, Harun .
JOURNAL OF INDUSTRIAL AND PRODUCTION ENGINEERING, 2016, 33 (06) :391-403
[4]   The two stage assembly flowshop scheduling problem to minimize total tardiness [J].
Allahverdi, Ali ;
Aydilek, Harun .
JOURNAL OF INTELLIGENT MANUFACTURING, 2015, 26 (02) :225-237
[5]   Bi-level genetic algorithms for a two-stage assembly flow-shop scheduling problem with batch delivery system [J].
Basir, Saeedeh Ahmadi ;
Mazdeh, Mohammad Mandavi ;
Namakshenas, Mohammad .
COMPUTERS & INDUSTRIAL ENGINEERING, 2018, 126 :217-231
[6]   Analyzing the effect of dispatching rules on the scheduling performance through grammar based flexible scheduling system [J].
Baykasoglu, Adil ;
Ozbakir, Lale .
INTERNATIONAL JOURNAL OF PRODUCTION ECONOMICS, 2010, 124 (02) :369-381
[7]   A novel shuffled frog-leaping algorithm with reinforcement learning for distributed assembly hybrid flow shop scheduling [J].
Cai, Jingcao ;
Lei, Deming ;
Wang, Jing ;
Wang, Lei .
INTERNATIONAL JOURNAL OF PRODUCTION RESEARCH, 2023, 61 (04) :1233-1251
[8]   A mathematical model and extension algorithm for assembly flexible flow shop scheduling problem [J].
Fattahi, Parviz ;
Hosseini, Seyed Mohammad Hassan ;
Jolai, Fariborz .
INTERNATIONAL JOURNAL OF ADVANCED MANUFACTURING TECHNOLOGY, 2013, 65 (5-8) :787-802
[9]   Multi-Objective Flexible Flow Shop Production Scheduling Problem Based on the Double Deep Q-Network Algorithm [J].
Gong, Hua ;
Xu, Wanning ;
Sun, Wenjuan ;
Xu, Ke .
PROCESSES, 2023, 11 (12)
[10]   Models and algorithms for production planning, scheduling and sequencing problems: A holistic framework and a systematic review [J].
Guzman, Eduardo ;
Andres, Beatriz ;
Poler, Raul .
JOURNAL OF INDUSTRIAL INFORMATION INTEGRATION, 2022, 27