Network parameter setting for reinforcement learning approaches using neural networks

Reinforcement learning approaches attract attention as the technique to construct the mapping function between sensors-motors of an autonomous robot through trial-and-error. Traditional reinforcement learning approaches make use of look-up table to express the mapping function between the grid state space and the grid action space. However the grid size of the state space affects the learning performances significantly. To overcome this problem, many researchers have proposed algorithms using neural networks to express the mapping function between the continuous state space and actions. However, in this case, a designer needs to appropriately set the number of middle neurons and the initial value of weight parameters of neural networks to improve the approximate accuracy of neural networks. This paper proposes a new method to automatically set the number of middle neurons and the initial value of the weight parameters of neural networks, on the basis of the dimensional-number of the sensor space, in Q-learning using neural networks. The proposed method is demonstrated through a navigation problem of an autonomous mobile robot, and is evaluated by comparing Q-learning using RBF networks and Q-learning using neural networks whose parameters are set by a designer. © 2012 The Japan Society of Mechanical Engineers.