Attention-Based Fault-Tolerant Approach for Multi-Agent Reinforcement Learning Systems

被引:9
|
作者
Gu, Shanzhi [1 ]
Geng, Mingyang [1 ]
Lan, Long [2 ]
机构
[1] Natl Univ Def Technol, Coll Comp, Changsha 410073, Peoples R China
[2] Natl Univ Def Technol, High Performance Comp Lab, Changsha 410073, Peoples R China
基金
中国国家自然科学基金;
关键词
reinforcement learning; attention mechanism; fault tolerance; multi-agent;
D O I
10.3390/e23091133
中图分类号
O4 [物理学];
学科分类号
0702 ;
摘要
The aim of multi-agent reinforcement learning systems is to provide interacting agents with the ability to collaboratively learn and adapt to the behavior of other agents. Typically, an agent receives its private observations providing a partial view of the true state of the environment. However, in realistic settings, the harsh environment might cause one or more agents to show arbitrarily faulty or malicious behavior, which may suffice to allow the current coordination mechanisms fail. In this paper, we study a practical scenario of multi-agent reinforcement learning systems considering the security issues in the presence of agents with arbitrarily faulty or malicious behavior. The previous state-of-the-art work that coped with extremely noisy environments was designed on the basis that the noise intensity in the environment was known in advance. However, when the noise intensity changes, the existing method has to adjust the configuration of the model to learn in new environments, which limits the practical applications. To overcome these difficulties, we present an Attention-based Fault-Tolerant (FT-Attn) model, which can select not only correct, but also relevant information for each agent at every time step in noisy environments. The multihead attention mechanism enables the agents to learn effective communication policies through experience concurrent with the action policies. Empirical results showed that FT-Attn beats previous state-of-the-art methods in some extremely noisy environments in both cooperative and competitive scenarios, much closer to the upper-bound performance. Furthermore, FT-Attn maintains a more general fault tolerance ability and does not rely on the prior knowledge about the noise intensity of the environment.
引用
收藏
页数:15
相关论文
共 50 条
  • [21] Distributed Fault-Tolerant Consensus Protocol for Fuzzy Multi-Agent Systems
    Chen, Shun
    Chen, Bo
    Shi, Feng
    CIRCUITS SYSTEMS AND SIGNAL PROCESSING, 2019, 38 (02) : 611 - 624
  • [22] Output Feedback Fault-Tolerant Control of Heterogeneous Multi-Agent Systems
    Yadegar, Meysam
    Meskin, Nader
    ASIAN JOURNAL OF CONTROL, 2021, 23 (02) : 949 - 961
  • [23] Fault-tolerant control of heterogeneous multi-agent systems with state constraints
    Jia, Jiyang
    Wang, Hao
    Liu, Lei
    INTERNATIONAL JOURNAL OF ADAPTIVE CONTROL AND SIGNAL PROCESSING, 2023, 37 (06) : 1592 - 1605
  • [24] Fault-Tolerant Optimal Consensus Control for Heterogeneous Multi-Agent Systems
    Li, Yandong
    Liu, Yongan
    Zhu, Ling
    Zhang, Zehua
    APPLIED SCIENCES-BASEL, 2024, 14 (16):
  • [25] Fault-tolerant tracking control for a class of nonlinear multi-agent systems
    Li, Xiayang
    Wang, Jinzhi
    SYSTEMS & CONTROL LETTERS, 2020, 135
  • [26] Fault-tolerant consensus for a network of multi-agent systems with actuator faults
    Department of Automation, Tsinghua University, Beijing
    100084, China
    Shanghai Jiaotong Daxue Xuebao, 6 (806-811):
  • [27] Observer-based Adaptive Fault-tolerant Consensus Tracking of Multi-agent Systems
    Zhao, Dan
    Fu, Junjie
    2019 CHINA-QATAR INTERNATIONAL WORKSHOP ON ARTIFICIAL INTELLIGENCE AND APPLICATIONS TO INTELLIGENT MANUFACTURING (AIAIM), 2019, : 30 - 35
  • [28] Fault-tolerant consensus control for multi-agent systems with actuator saturation
    Gao, Chen
    He, Xiao
    PROCEEDINGS 2018 33RD YOUTH ACADEMIC ANNUAL CONFERENCE OF CHINESE ASSOCIATION OF AUTOMATION (YAC), 2018, : 484 - 488
  • [29] Fault-tolerant Consensus of Leader-follower Multi-agent Systems
    Zhao, Dan
    Wu, Yonghong
    Yang, Chao
    Ling, Guang
    Shen, Yanjun
    Guan, Zhi-Hong
    PROCEEDINGS OF THE 28TH CHINESE CONTROL AND DECISION CONFERENCE (2016 CCDC), 2016, : 3288 - 3293
  • [30] Fault-Tolerant Containment Control for Linear Multi-Agent Systems: An Adaptive Output Regulation Approach
    Zhang, Jie
    Ding, Da-Wei
    An, Cuijuan
    IEEE ACCESS, 2019, 7 : 89306 - 89315