Rapid behavior learning in multi-agent environment based on state value estimation of others

被引:0
|
作者
Takahashi, Yasutake [1 ]
Noma, Kentaro [1 ]
Asada, Minoru [1 ]
机构
[1] Osaka Univ, Dept Adapt Machine Syst, Suita, Osaka 5650871, Japan
来源
2007 IEEE/RSJ INTERNATIONAL CONFERENCE ON INTELLIGENT ROBOTS AND SYSTEMS, VOLS 1-9 | 2007年
关键词
D O I
10.1109/IROS.2007.4399294
中图分类号
TP [自动化技术、计算机技术];
学科分类号
0812 ;
摘要
The existing reinforcement learning approaches have been suffering from the curse of dimension problem when they are applied to multiagent dynamic environments. One of the typical examples is a case of RoboCup competitions since other agents and their behaviors easily cause state and action space explosion. This paper presents a method of modular learning in a multiagent environment by which the learning agent can acquire cooperative behaviors with its team mates and competitive ones against its opponents. The key ideas to resolve the issue are as follows. First, a two-layer hierarchical system with multi learning modules is adopted to reduce the size of the sensor and action spaces. The state space of the top layer consists of the state values from the lower level, and the macro actions are used to reduce the size of the physical action space. Second, the state of the other to what extent it is close to its own goal is estimated by observation and used as a state value in the top layer state space to realize the cooperative/competitive behaviors. The method is applied to 4 (defense team) on 5 (offense team) game task, and the learning agent successfully acquired the teamwork plays (pass and shoot) within much shorter learning time (30 times quicker than the earlier work).
引用
收藏
页码:76 / 81
页数:6
相关论文
共 50 条
  • [31] State-based episodic memory for multi-agent reinforcement learning
    Ma, Xiao
    Li, Wu-Jun
    MACHINE LEARNING, 2023, 112 (12) : 5163 - 5190
  • [32] Simulation and multi-agent environment for aircraft maintenance learning
    Gouardères, G
    Minko, A
    Richard, L
    ARTIFICIAL INTELLIGENCE: METHODOLOGY, SYSTEMS, APPLICATIONS, PROCEEDINGS, 2000, 1904 : 152 - 166
  • [33] Battlefield Environment Design for Multi-agent Reinforcement Learning
    Do, Seungwon
    Baek, Jaeuk
    Jun, Sungwoo
    Lee, Changeun
    2022 IEEE INTERNATIONAL CONFERENCE ON BIG DATA AND SMART COMPUTING (IEEE BIGCOMP 2022), 2022, : 318 - 319
  • [34] Studies on hierarchical reinforcement learning in multi-agent environment
    Yu Lasheng
    Marin, Alonso
    Hong Fei
    Lin Jian
    PROCEEDINGS OF 2008 IEEE INTERNATIONAL CONFERENCE ON NETWORKING, SENSING AND CONTROL, VOLS 1 AND 2, 2008, : 1714 - 1720
  • [35] Learning Efficient Communication in Cooperative Multi-Agent Environment
    Zhao, Yuhang
    Ma, Xiujun
    AAMAS '19: PROCEEDINGS OF THE 18TH INTERNATIONAL CONFERENCE ON AUTONOMOUS AGENTS AND MULTIAGENT SYSTEMS, 2019, : 2321 - 2323
  • [36] Q-LEARNING BY THE nth STEP STATE AND MULTI-AGENT NEGOTIATION IN UNKNOWN ENVIRONMENT
    Job, Josip
    Jovic, Franjo
    Livada, Caslav
    TEHNICKI VJESNIK-TECHNICAL GAZETTE, 2012, 19 (03): : 529 - 534
  • [37] Decentralized State Estimation for Heterogeneous Multi-Agent Systems
    Boem, Francesca
    Sabattini, Lorenzo
    Secchi, Cristian
    2015 54TH IEEE CONFERENCE ON DECISION AND CONTROL (CDC), 2015, : 4121 - 4126
  • [38] Multi-agent learning and the descriptive value of simple models
    Erev, Ido
    Roth, Alvin E.
    ARTIFICIAL INTELLIGENCE, 2007, 171 (07) : 423 - 428
  • [39] Deconfounded Value Decomposition for Multi-Agent Reinforcement Learning
    Li, Jiahui
    Kuang, Kun
    Wang, Baoxiang
    Liu, Furui
    Chen, Long
    Fan, Changjie
    Wu, Fei
    Xiao, Jun
    INTERNATIONAL CONFERENCE ON MACHINE LEARNING, VOL 162, 2022,
  • [40] Collaborative design environment based on multi-agent
    Chen, SL
    Wu, HZ
    Han, XL
    Xiao, L
    PROCEEDINGS OF THE NINTH INTERNATIONAL CONFERENCE ON COMPUTER SUPPORTED COOPERATIVE WORK IN DESIGN, VOLS 1 AND 2, 2005, : 481 - 485