Anti-conflict AGV path planning in automated container terminals based on multi-agent reinforcement learning

被引：91

作者：

Hu, Hongtao ^{[1
]}

Yang, Xurui ^{[2
]}

Xiao, Shichang ^{[1
]}

Wang, Feiyang ^{[1
]}

机构：

[1] Shanghai Maritime Univ, Sch Logist Engn, Shanghai, Peoples R China

[2] Shanghai Maritime Univ, Inst Logist Sci & Engn, Shanghai 201306, Peoples R China

来源：

INTERNATIONAL JOURNAL OF PRODUCTION RESEARCH | 2023年 / 61卷 / 01期

基金：

中国国家自然科学基金;

关键词：

Automated container terminal; AGV path planning; Anti-conflict; reinforcement learning; policy gradient; GUIDED VEHICLES; PREVENTION; SYSTEMS;

D O I：

10.1080/00207543.2021.1998695

中图分类号：

T [工业技术];

学科分类号：

08 ;

摘要：

AGV conflict prevention path planning is a key factor to improve transportation cost and operation efficiency of the container terminal. This paper studies the anti-conflict path planning problem of Automated Guided Vehicle (AGV) in the horizontal transportation area of the Automated Container Terminals (ACTs). According to the characteristics of magnetic nail guided AGVs, a node network is constructed. Through the analysis of two conflict situations, namely the opposite conflict situation and same point occupation conflict situation, an integer programming model is established to obtain the shortest path. The Multi-Agent Deep Deterministic Policy Gradient (MADDPG) method is proposed to solve the problem, and the Gumbel-Softmax strategy is applied to discretize the scenario created by the node network. A series of numerical experiments are conducted to verify the effectiveness and the efficiency of the model and the algorithm.

引用

页码：65 / 80

页数：16

共 39 条

[31] Scheduling and routing algorithms for AGVs: a survey [J].

Qiu, L ;

Hsu, WJ ;

Huang, SY ;

Wang, H .

INTERNATIONAL JOURNAL OF PRODUCTION RESEARCH, 2002, 40 (03) :745-760

[32]

Samvelyan M, 2019, AAMAS '19: PROCEEDINGS OF THE 18TH INTERNATIONAL CONFERENCE ON AUTONOMOUS AGENTS AND MULTIAGENT SYSTEMS, P2186

[33]

Sutton R., 2014, Reinforcement Learning: An Introduction

[34] ON THE THEORY OF THE BROWNIAN MOTION [J].

TODA, M .

JOURNAL OF THE PHYSICAL SOCIETY OF JAPAN, 1958, 13 (11) :1266-1280

[35]

Vinitsky Eugene, Benchmarks for reinforcement learning in mixed-autonomy traffic

[36]

WATKINS CJCH, 1992, MACH LEARN, V8, P279, DOI 10.1007/BF00992698

[37] Multi-vehicle routing problems with soft time windows: A multi-agent reinforcement learning approach [J].

Zhang, Ke ;

He, Fang ;

Zhang, Zhengchao ;

Lin, Xi ;

Li, Meng .

TRANSPORTATION RESEARCH PART C-EMERGING TECHNOLOGIES, 2020, 121

[38] Energy-efficient path planning for a single-load automated guided vehicle in a manufacturing workshop [J].

Zhang, Zhongwei ;

Wu, Lihui ;

Zhang, Wenqiang ;

Peng, Tao ;

Zheng, Jun .

COMPUTERS & INDUSTRIAL ENGINEERING, 2021, 158

[39] Modeling of yard congestion and optimization of yard template in container ports [J].

Zhen, Lu .

TRANSPORTATION RESEARCH PART B-METHODOLOGICAL, 2016, 90 :83-104

← 1 2 3 4 →