Q-learning in Continuous State-Action Space with Redundant Dimensions by Using a Selective Desensitization Neural Network

When applying reinforcement learning algorithms such as Q-learning to real world problems, we must consider the high and redundant dimensions and continuity of the state-action space. The continuity of state-action space is often treated by value function approximation. However, conventional function approximators such as radial basis function networks (RBFNs) are unsuitable in these environments, because they incur high computational cost, and the number of required experiences grows exponentially with the dimension of the state-action space. By contrast, selective desensitization neural network (SDNN) is highly robust to redundant inputs and computes at low computational cost. This paper proposes a novel function approximation method for Q-learning in continuous state-action space based on SDNN. The proposed method is evaluated by numerical experiments with redundant input(s). These experimental results validate the robustness of the proposed method to redundant state dimensions, and its lower computational cost than RBFN. These properties are advantageous to real-world applications such as robotic systems.