A Safe Reinforcement Learning driven Weights-varying Model Predictive Control for Autonomous Vehicle Motion Control

被引:0
作者
Zarrouki, Baha [1 ,2 ]
Spanakakis, Marios
Betz, Johannes
机构
[1] Tech Univ Munich, TUM Sch Engn & Design, Automot Technol, Munich, Germany
[2] Tech Univ Munich, TUM Sch Engn & Design, Autonomous Vehicle Syst, Munich, Germany
来源
2024 35TH IEEE INTELLIGENT VEHICLES SYMPOSIUM, IEEE IV 2024 | 2024年
关键词
MPC;
D O I
10.1109/IV55156.2024.10588747
中图分类号
TP [自动化技术、计算机技术];
学科分类号
0812 ;
摘要
Determining the optimal cost function parameters of Model Predictive Control (MPC) to optimize multiple control objectives is a challenging and time-consuming task. Multi-objective Bayesian Optimization (BO) techniques solve this problem by determining a Pareto optimal parameter set for an MPC with static weights. However, a single parameter set may not deliver the most optimal closed-loop control performance when the context of the MPC operating conditions changes during its operation, urging the need to adapt the cost function weights at runtime. Deep Reinforcement Learning (RL) algorithms can automatically learn context-dependent optimal parameter sets and dynamically adapt for a Weights-varying MPC (WMPC). However, learning cost function weights from scratch in a continuous action space may lead to unsafe operating states. To solve this, we propose a novel approach limiting the RL action space within a safe learning space that we represent by a catalog of pre-optimized feasible BO Pareto-optimal weight sets. We conceive an RL agent not to learn in a continuous space but to select the most optimal discrete actions, each corresponding to a single set of Pareto optimal weights, by proactively anticipating upcoming control tasks in a context-dependent manner. This approach introduces a two-step optimization: (1) safety-critical with BO and (2) performance-driven with RL. Hence, even an untrained RL agent guarantees a safe and optimal performance. Simulation results demonstrate that an untrained RL-WMPC shows Pareto-optimal closed-loop behavior and training the RL-WMPC helps exhibit a performance beyond the Pareto-front. The code used in this research is publicly accessible as open-source software: https://github.com/bzarr/TUM-CONTROL
引用
收藏
页码:1401 / 1408
页数:8
相关论文
共 50 条
  • [1] Prediction Horizon-Varying Model Predictive Control (MPC) for Autonomous Vehicle Control
    Chen, Zhenbin
    Lai, Jiaqin
    Li, Peixin
    Awad, Omar I.
    Zhu, Yubing
    [J]. ELECTRONICS, 2024, 13 (08)
  • [2] Model predictive control for autonomous underwater vehicle
    Budiyono, Agus
    [J]. INDIAN JOURNAL OF GEO-MARINE SCIENCES, 2011, 40 (02) : 191 - 199
  • [3] Reinforcement Learning-based Event-Triggered Model Predictive Control for Autonomous Vehicle Path Following
    Chen, Jun
    Meng, Xiangyu
    Li, Zhaojian
    [J]. 2022 AMERICAN CONTROL CONFERENCE, ACC, 2022, : 3342 - 3347
  • [4] Autonomous navigation at unsignalized intersections: A coupled reinforcement learning and model predictive control approach
    Bautista-Montesano, Rolando
    Galluzzi, Renato
    Ruan, Kangrui
    Fu, Yongjie
    Di, Xuan
    [J]. TRANSPORTATION RESEARCH PART C-EMERGING TECHNOLOGIES, 2022, 139
  • [5] Safe hierarchical model predictive control and planning for autonomous systems
    Koegel, Markus
    Ibrahim, Mohamed
    Kallies, Christian
    Findeisen, Rolf
    [J]. INTERNATIONAL JOURNAL OF ROBUST AND NONLINEAR CONTROL, 2025, 35 (07) : 2658 - 2676
  • [6] Model-free Data-driven Predictive Control Using Reinforcement Learning
    Sawant, Shambhuraj
    Reinhardt, Dirk
    Kordabad, Arash Bahari
    Gros, Sebastien
    [J]. 2023 62ND IEEE CONFERENCE ON DECISION AND CONTROL, CDC, 2023, : 4046 - 4052
  • [7] Learning Model Predictive Control for Connected Autonomous Vehicles
    Jafarzadeh, Hassan
    Fleming, Cody
    [J]. 2019 IEEE 58TH CONFERENCE ON DECISION AND CONTROL (CDC), 2019, : 2336 - 2343
  • [8] Reinforcement learning and model predictive control for robust embedded quadrotor guidance and control
    Greatwood, Colin
    Richards, Arthur G.
    [J]. AUTONOMOUS ROBOTS, 2019, 43 (07) : 1681 - 1693
  • [9] Horizonwise Model-Predictive Control With Application to Autonomous Driving Vehicle
    Choi, Woo Young
    Lee, Seung-Hi
    Chung, Chung Choo
    [J]. IEEE TRANSACTIONS ON INDUSTRIAL INFORMATICS, 2022, 18 (10) : 6940 - 6949
  • [10] Safe Stochastic Model Predictive Control
    Brudigam, T.
    Jacumet, R.
    Wollherr, D.
    Leibold, M.
    [J]. 2022 IEEE 61ST CONFERENCE ON DECISION AND CONTROL (CDC), 2022, : 1796 - 1802