Adaptability analysis of genetic network programming with reinforcement learning in dynamically changing environments

被引:7
作者
Mabu, Shingo [1 ]
Tjahjadi, Andre [1 ]
Hirasawa, Kotaro [1 ]
机构
[1] Waseda Univ, Grad Sch Informat Prod & Syst, Wakamatsu Ku, Kitakyushu, Fukuoka 8080135, Japan
关键词
Evolutionary computation; Genetic network programming; Reinforcement learning; Adaptability; Khepera robot; EVOLUTION;
D O I
10.1016/j.eswa.2012.04.038
中图分类号
TP18 [人工智能理论];
学科分类号
081104 ; 0812 ; 0835 ; 1405 ;
摘要
Genetic network programming (GNP) has been proposed as one of the evolutionary algorithms and extended with reinforcement learning (GNP-RL). The combination of evolution and learning can efficiently evolve programs and the fitness improvement has been confirmed in the simulations of tileworld problems, elevator group supervisory control systems, stock trading models and wall following behavior of Khepera robot. However, its adaptability in testing environments, where the situations dynamically change, has not been analyzed in detail yet. In this paper, the adaptation mechanism in the testing environment is introduced and it is confirmed that GNP-RL can adapt to the environmental changes using a robot simulator WEBOTS, especially when unexperienced sensor troubles suddenly occur. The simulation results show that GNP-RL works well in the testing even if wrong sensor information is given because GNP-RL has a function to automatically change programs using alternative actions. In addition, the analysis on the effects of the parameters of GNP-RL is carried out in both training and testing simulations. (C) 2012 Elsevier Ltd. All rights reserved.
引用
收藏
页码:12349 / 12357
页数:9
相关论文
共 11 条
[1]  
[Anonymous], 1994, Genetic programming II: Automatic discovery of reusable programs, DOI DOI 10.5555/183460
[2]  
[Anonymous], 2003, Genetic programming IV: routine human-competitive machine intelligence
[3]  
FLOREANO D, 1994, COM ADAP SY, P421
[4]   A double-deck elevator group supervisory control system using genetic network programming [J].
Hirasawa, Kotaro ;
Eguchi, Toru ;
Zhou, Jin ;
Yu, Lu ;
Hu, Jinglu ;
Markon, Sandor .
IEEE TRANSACTIONS ON SYSTEMS MAN AND CYBERNETICS PART C-APPLICATIONS AND REVIEWS, 2008, 38 (04) :535-550
[5]   Adaptation technique for integrating genetic programming and reinforcement learning for real robots [J].
Kamio, S ;
Iba, H .
IEEE TRANSACTIONS ON EVOLUTIONARY COMPUTATION, 2005, 9 (03) :318-333
[6]  
Mabu S., 2010, 2010 IEEE International Conference on Systems, Man and Cybernetics (SMC 2010), P1659, DOI 10.1109/ICSMC.2010.5642323
[7]  
Mabu S., 2006, Transactions of the Institute of Electrical Engineers of Japan, Part C, V126-C, P1009, DOI 10.1541/ieejeiss.126.1009
[8]   A graph-based evolutionary algorithm: Genetic Network Programming (GNP) and its extension using reinforcement learning [J].
Mabu, Shingo ;
Hirasawa, Kotaro ;
Hu, Jinglu .
EVOLUTIONARY COMPUTATION, 2007, 15 (03) :369-398
[9]  
Mabu S, 2007, IEEE C EVOL COMPUTAT, P508
[10]   Evolution of a world model for a miniature robot using genetic programming [J].
Nordin, P ;
Banzhaf, W ;
Brameier, M .
ROBOTICS AND AUTONOMOUS SYSTEMS, 1998, 25 (1-2) :105-116