Identifiability in training neural networks for reconfigurable control based on reinforcement learning

The field of reconfigurable control has become more and more active in the last few years. Many control strategies for constructing an autonomous adapting control system have been developed in the past. However, many of those strategies require a Failure Detection and Isolation (FDI) system such that the control laws can be adapted properly. The major drawback of an FDI system is that the designer needs to foresee all possible failure scenarios, something virtually impossible for even simple systems. In this paper, a reconfigurable control system based on Reinforcement Learning (RL) is proposed. Reinforcement Learning does not require a FDI system, which makes it ideal for reconfigurable control and for controlling unknown plants. Neural networks are used to solve the 'curse of dimensionality' inherent to a RL controller. A novel method of batch updating in combination with a new training algorithm based on the Newton-Gauss method are applied to solve two major problems of neural networks, i.e. the 'recency'-effect and the identifiability problem. Closely related to the identifiability problem is the optimization of the neural network structure. To guarantee the optimal amount of network parameters an optimization algorithm is proposed. The techniques are implemented on a simple system identification and a pole balancing task. Experiments show that the use of,anti-recency' points circumvent the recency-effect in case of system identification and speed up stabilization for the RL controller.