Resilient reinforcement learning and robust output regulation under denial-of-service attacks

被引:70
作者
Gao, Weinan [1 ,2 ]
Deng, Chao [3 ]
Jiang, Yi [1 ,4 ]
Jiang, Zhong-Ping [5 ]
机构
[1] Northeastern Univ, State Key Lab Synthet Automat Proc Ind, Shenyang 110004, Peoples R China
[2] Florida Inst Technol, Dept Mech & Civil Engn, Melbourne, FL 32901 USA
[3] Nanjing Univ Posts & Telecommun, Inst Adv Technol, Nanjing 210023, Peoples R China
[4] City Univ Hong Kong, Dept Biomed Engn, Hong Kong, Peoples R China
[5] NYU, 6 MetroTech Ctr, Dept Elect & Comp Engn, Brooklyn, NY 11201 USA
基金
美国国家科学基金会;
关键词
Reinforcement learning; Robust output regulation; Hybrid iteration; Denial-of-service attacks; ADAPTIVE OPTIMAL-CONTROL; NETWORKED CONTROL; LINEAR-SYSTEMS; STABILITY; FRAMEWORK; ITERATION; INPUT;
D O I
10.1016/j.automatica.2022.110366
中图分类号
TP [自动化技术、计算机技术];
学科分类号
0812 ;
摘要
In this paper, we have proposed a novel resilient reinforcement learning approach for solving robust optimal output regulation problems of a class of partially linear systems under both dynamic uncertainties and denial-of-service attacks. Fundamentally different from existing works on reinforcement learning, the proposed approach rigorously analyzes both the resilience of closed-loop systems against attacks and the robustness against dynamic uncertainties. Moreover, we have proposed an original successive approximation approach, named hybrid iteration, to learn the robust optimal control policy, that converges faster than value iteration, and is independent of an initial admissible controller. Simulation results demonstrate the efficacy of the proposed approach. (C) 2022 Elsevier Ltd. All rights reserved.
引用
收藏
页数:9
相关论文
共 39 条
[1]  
Amin S, 2009, LECT NOTES COMPUT SC, V5469, P31, DOI 10.1007/978-3-642-00602-9_3
[2]   Decentralized Adaptive Fuzzy Secure Control for Nonlinear Uncertain Interconnected Systems Against Intermittent DoS Attacks [J].
An, Liwei ;
Yang, Guang-Hong .
IEEE TRANSACTIONS ON CYBERNETICS, 2019, 49 (03) :827-838
[3]   CONTINUOUS-TIME ROBUST DYNAMIC PROGRAMMING [J].
Bian, Tao ;
Jiang, Zhong-Ping .
SIAM JOURNAL ON CONTROL AND OPTIMIZATION, 2019, 57 (06) :4150-4174
[4]   Value iteration and adaptive dynamic programming for data-driven adaptive optimal control design [J].
Bian, Tao ;
Jiang, Zhong-Ping .
AUTOMATICA, 2016, 71 :348-360
[5]   Input-to-State Stabilizing Control Under Denial-of-Service [J].
De Persis, Claudio ;
Tesi, Pietro .
IEEE TRANSACTIONS ON AUTOMATIC CONTROL, 2015, 60 (11) :2930-2944
[6]   Distributed Resilient Observer-Based Fault-Tolerant Control for Heterogeneous Multiagent Systems Under Actuator Faults and DoS Attacks [J].
Deng, Chao ;
Wen, Changyun .
IEEE TRANSACTIONS ON CONTROL OF NETWORK SYSTEMS, 2020, 7 (03) :1308-1318
[7]   Networked Control Under DoS Attacks: Tradeoffs Between Resilience and Data Rate [J].
Feng, Shuai ;
Cetinkaya, Ahmet ;
Ishii, Hideaki ;
Tesi, Pietro ;
De Persis, Claudio .
IEEE TRANSACTIONS ON AUTOMATIC CONTROL, 2021, 66 (01) :460-467
[8]   Resilient control under Denial-of-Service: Robust design [J].
Feng, Shuai ;
Tesi, Pietro .
AUTOMATICA, 2017, 79 :42-51
[9]   Extremum Seeking Under Persistent Gradient Deception: A Switching Systems Approach [J].
Galarza-Jimenez, Felipe ;
Poveda, Jorge I. ;
Bianchin, Gianluca ;
Dall'Anese, Emiliano .
IEEE CONTROL SYSTEMS LETTERS, 2022, 6 :133-138
[10]   Learning-based adaptive optimal output regulation of linear and nonlinear systems: an overview [J].
Gao, Weinan ;
Jiang, Zhong-Ping .
CONTROL THEORY AND TECHNOLOGY, 2022, 20 (01) :1-19