Local and soft feature selection for value function approximation in batch reinforcement learning for robot navigation

被引:0
作者
Fathinezhad, Fatemeh [1 ]
Adibi, Peyman [1 ]
Shoushtarian, Bijan [1 ]
Chanussot, Jocelyn [2 ]
机构
[1] Univ Isfahan, Fac Comp Engn, Artificial Intelligence Dept, Esfahan, Iran
[2] Univ Grenoble Alpes, Grenoble INP, GIPSA Lab, CNRS, Grenoble, France
基金
美国国家科学基金会;
关键词
Reinforcement learning; Value function approximation; Local relevance feature selection; Robot navigation; SQUARES POLICY ITERATION; FRAMEWORK; ONLINE;
D O I
10.1007/s11227-023-05854-4
中图分类号
TP3 [计算技术、计算机技术];
学科分类号
0812 ;
摘要
This paper proposes a novel method for robot navigation in high-dimensional environments that reduce the dimension of the state space using local and soft feature selection. The algorithm selects relevant features based on local correlations between states, avoiding duplicate inappropriate information and adjusting sensor values accordingly. By optimizing the value function approximation based on the local weighted features of states in the reinforcement learning process, the method shows improvements in the robot's motion flexibility, learning time, the distance traveled to reach its goal, and the minimization of collisions with obstacles. This approach was tested on an E-puck robot using the Webots robot simulator in different test environments.
引用
收藏
页码:10720 / 10745
页数:26
相关论文
共 33 条
[1]   Infinite-horizon policy-gradient estimation [J].
Baxter, J ;
Bartlett, PL .
JOURNAL OF ARTIFICIAL INTELLIGENCE RESEARCH, 2001, 15 :319-350
[2]  
Bertsekas DP, 1995, PROCEEDINGS OF THE 34TH IEEE CONFERENCE ON DECISION AND CONTROL, VOLS 1-4, P560, DOI 10.1109/CDC.1995.478953
[3]  
Boyan JA, 1999, MACHINE LEARNING, PROCEEDINGS, P49
[4]  
Cover TM., 2006, ELEMENTS INFORM THEO, DOI [DOI 10.1002/047174882X, DOI 10.1002/0471200611.CH2]
[5]   Kernel dynamic policy programming: Applicable reinforcement learning to robot systems with high dimensional states [J].
Cui, Yunduan ;
Matsubara, Takamitsu ;
Sugimoto, Kenji .
NEURAL NETWORKS, 2017, 94 :13-23
[6]   Normalized Mutual Information Feature Selection [J].
Estevez, Pablo. A. ;
Tesmer, Michel ;
Perez, Claudio A. ;
Zurada, Jacek A. .
IEEE TRANSACTIONS ON NEURAL NETWORKS, 2009, 20 (02) :189-201
[7]   Least-squares policy iteration algorithms for robotics: Online, continuous, and automatic [J].
Friedrich, Stefan R. ;
Schreibauer, Michael ;
Buss, Martin .
ENGINEERING APPLICATIONS OF ARTIFICIAL INTELLIGENCE, 2019, 83 :72-84
[8]   Algorithmic Survey of Parametric Value Function Approximation [J].
Geist, Matthieu ;
Pietquin, Olivier .
IEEE TRANSACTIONS ON NEURAL NETWORKS AND LEARNING SYSTEMS, 2013, 24 (06) :845-867
[9]   Clustering subspace generalization to obtain faster reinforcement learning [J].
Hashemzadeh, Maryam ;
Hosseini, Reshad ;
Ahmadabadi, Majid Nili .
EVOLVING SYSTEMS, 2020, 11 (01) :89-103
[10]  
Khalilullah KMI, 2018, 2018 57TH ANNUAL CONFERENCE OF THE SOCIETY OF INSTRUMENT AND CONTROL ENGINEERS OF JAPAN (SICE), P798, DOI 10.23919/SICE.2018.8492578