Traffic Light Control Using Hierarchical Reinforcement Learning and Options Framework

被引：12

作者：

Borges, Dimitrius F. ^{[1
]}

Leite, Joao Paulo R. R. ^{[1
]}

Moreira, Edmilson M. ^{[1
]}

Carpinteiro, Otavio A. S. ^{[1
]}

机构：

[1] Univ Fed Itajuba, Inst Syst Engn & Informat Technol, BR-1303 Itajuba, MG, Brazil

来源：

IEEE ACCESS | 2021年 / 9卷

关键词：

Reinforcement learning; Vehicle dynamics; Tools; Mathematical model; Meters; Green products; Adaptation models; Intelligent systems; machine learning; reinforcement learning; simulation; traffic control; SIGNAL CONTROL; INTELLIGENCE;

D O I：

10.1109/ACCESS.2021.3096666

中图分类号：

TP [自动化技术、计算机技术];

学科分类号：

0812 ;

摘要：

The number of vehicles worldwide has grown rapidly over the past decade, impacting how urban traffic is managed. Traffic light control is a well-known problem and, although an increasing number of technologies are used to solve it, it still poses challenges and opportunities, especially when considering the inefficiency of the popular fixed-time traffic controllers. This study aims to apply Hierarchical Reinforcement Learning (HRL) and Options Framework to control a signalized vehicular intersection and compare its performance with that of a fixed-time traffic controller, configured using the Webster Method. HRL combines the ability to learn and make decisions while taking observations from the environment in real-time. These capabilities bring a significant adaptive power to a highly dynamic problem. The test scenarios were built using the SUMO simulation tool. According to our results, HRL presents better performance than those of its own isolated sub-policies and the fixed-time model, indicating a simple and efficient alternative.

引用

页码：99155 / 99165

页数：11

共 37 条

[21]

Lapan M., 2018, Deep reinforcement learning hands-on: Apply modern RL methods, with deep Q-networks, value iteration, policy gradients, TRPO, AlphaGo Zero and more

[22]

Larson R., 2015, Elementary statistics: Picturing the world

[23]

Lopez PA, 2018, IEEE INT C INTELL TR, P2575, DOI 10.1109/ITSC.2018.8569938

[24]

Machado MC, 2017, PR MACH LEARN RES, V70

[25]

Markov A., 1907, DYNAMIC PROBABILISTI, V1, P552

[26]

Prabuchandran KJ, 2014, 2014 IEEE 17TH INTERNATIONAL CONFERENCE ON INTELLIGENT TRANSPORTATION SYSTEMS (ITSC), P2529, DOI 10.1109/ITSC.2014.6958095

[27] TRAFFIC CONTROL USING GRAPH-THEORY [J].

RIEDEL, T ;

BRUNNER, U .

CONTROL ENGINEERING PRACTICE, 1994, 2 (03) :397-404

[28] Optimal Behavioral Hierarchy [J].

Solway, Alec ;

Diuk, Carlos ;

Cordova, Natalia ;

Yee, Debbie ;

Barto, Andrew G. ;

Niv, Yael ;

Botvinick, Matthew M. .

PLOS COMPUTATIONAL BIOLOGY, 2014, 10 (08)

[29]

SUMO, 2020, SIMULATION URBAN MOB

[30]

Sutton Richard S., 2015, Reinforcement Learning: An Introduction

← 1 2 3 4 →