DAVION LEAST SQUARES-BASED LEARNING ALGORITHM FOR FEEDFORWARD NEURAL NETWORKS

This paper presents a new learning methodology for feedforward neural networks. The proposed algorithm is based on Davidon's least squares minimization approach. The performance of the Davidon algorithm is compared with that of the back propagation for three prototype examples. These examples are: Case I, a linear second-order system; Case II, a chaotic system; and Case III, a nonlinear dynamic system. The trained network is employed to determine the one-step-ahead prediction of the output of a given system. The simulation results show that, in most cases, the Davidon algorithm has an order of magnitude faster convergence rate than that of the back propagation. In order to make a fair comparison, an optimum back propagation learning rate is used. The learning rate chosen is the one that results in the fastest convergence for a given system.