Genetic Programming discovers efficient learning rules for the hidden and output layers of feedforward neural networks

The learning method is critical for obtaining good generalisation in neural networks with limited training data. The Standard BackPropagation (SBP) training algorithm suffers from several problems such as sensitivity to the initial conditions and very slow convergence. The aim of this work is to use Genetic Programming; (GP) to discover new supervised learning algorithms which can overcome some of these problems. In previous research a new learning algorithms for the output layer has been discovered using GP. By comparing this with SEP on different problems better performance was demonstrated. This paper shows that GP can also discover better learning algorithms for the hidden layers to be used in conjunction with the algorithm previously discovered. Comparing these with SEP on different problems we show they provide better performances. This study indicates that there exist many supervised learning algorithms better than SEP and that GP can be used to discover them.