Reinforcement Learning Optimized Look-Ahead Energy Management of a Parallel Hybrid Electric Vehicle

被引:333
作者
Liu, Teng [1 ]
Hu, Xiaosong [2 ,3 ]
Li, Shengbo Eben [4 ]
Cao, Dongpu [5 ]
机构
[1] Chinese Acad Sci, State Key Lab Management & Control Complex Syst, Inst Automat, Beijing 100864, Peoples R China
[2] Chongqing Univ, State Key Lab Mech Transmiss, Chongqing 400044, Peoples R China
[3] Chongqing Univ, Dept Automot Engn, Chongqing 400044, Peoples R China
[4] Tsinghua Univ, Dept Automot Engn, State Key Lab Automot Safety & Energy, Beijing 100084, Peoples R China
[5] Cranfield Univ, Ctr Automot Engn, Bedford MK43 0AL, England
关键词
Energy management; hybrid electric vehicle (HEV); Markov chain (MC); predictive control; reinforcement learning (RL); MODEL-PREDICTIVE CONTROL; ECMS;
D O I
10.1109/TMECH.2017.2707338
中图分类号
TP [自动化技术、计算机技术];
学科分类号
0812 ;
摘要
This paper presents a predictive energy management strategy for a parallel hybrid electric vehicle (HEV) based on velocity prediction and reinforcement learning (RL). The design procedure starts with modeling the parallel HEV as a systematic control-oriented model and defining a cost function. Fuzzy encoding and nearest neighbor approaches are proposed to achieve velocity prediction, and a finite-state Markov chain is exploited to learn transition probabilities of power demand. To determine the optimal control behaviors and power distribution between two energy sources, a novel RL-based energy management strategy is introduced. For comparison purposes, the two velocity prediction processes are examined by RL using the same realistic driving cycle. The look-ahead energy management strategy is contrasted with shortsighted and dynamic programming based counterparts, and further validated by hardware-in-the-loop test. The results demonstrate that the RL-optimized control is able to significantly reduce fuel consumption and computational time.
引用
收藏
页码:1497 / 1507
页数:11
相关论文
共 45 条
[1]  
[Anonymous], 2004, P 7 INT S ADV VEH CO
[2]  
[Anonymous], APPL ENERGY
[3]  
[Anonymous], IEEE T VEH IN PRESS
[4]   MPC-Based Energy Management of a Power-Split Hybrid Electric Vehicle [J].
Borhan, Hoseinali ;
Vahidi, Ardalan ;
Phillips, Anthony M. ;
Kuang, Ming L. ;
Kolmanovsky, Ilya V. ;
Di Cairano, Stefano .
IEEE TRANSACTIONS ON CONTROL SYSTEMS TECHNOLOGY, 2012, 20 (03) :593-603
[5]   Bioavailability of andrographolide and protection against carbon tetrachloride-induced oxidative damage in rats [J].
Chen, Haw-Wen ;
Huang, Chin-Shiu ;
Li, Chien-Chun ;
Lin, Ai-Hsuan ;
Huang, Yu-Ju ;
Wang, Tsu-Shing ;
Yao, Hsien-Tsung ;
Lii, Chong-Kuei .
TOXICOLOGY AND APPLIED PHARMACOLOGY, 2014, 280 (01) :1-9
[6]   Design and Experimental Evaluations on Energy Efficient Control Allocation Methods for Overactuated Electric Vehicles: Longitudinal Motion Case [J].
Chen, Yan ;
Wang, Junmin .
IEEE-ASME TRANSACTIONS ON MECHATRONICS, 2014, 19 (02) :538-548
[7]   Game Theory Controller for Hybrid Electric Vehicles [J].
Dextreit, Clement ;
Kolmanovsky, Ilya V. .
IEEE TRANSACTIONS ON CONTROL SYSTEMS TECHNOLOGY, 2014, 22 (02) :652-663
[8]   Recursive Markov Decision Processes and Recursive Stochastic Games [J].
Etessami, Kousha ;
Yannakakis, Mihalis .
JOURNAL OF THE ACM, 2015, 62 (02)
[9]   A Supervisory Control Strategy for Plug-In Hybrid Electric Vehicles Based on Energy Demand Prediction and Route Preview [J].
Feng Tianheng ;
Yang Lin ;
Gu Qing ;
Hu Yanqing ;
Yan Ting ;
Yan Bin .
IEEE TRANSACTIONS ON VEHICULAR TECHNOLOGY, 2015, 64 (05) :1691-1700
[10]   Generalized Markov Models for Real-Time Modeling of Continuous Systems [J].
Filev, Dimitar P. ;
Kolmanovsky, Ilya .
IEEE TRANSACTIONS ON FUZZY SYSTEMS, 2014, 22 (04) :983-998