An Improved Conjugate Gradient Neural Networks Based on a Generalized Armijo Search Method

In this paper, by constructing a generalized Armijo search method, a novel conjugate gradient (CG) model has been proposed to training a common three-layer backpropagation (BP) neural network. Compared with the classical gradient descent method, this algorithm efficiently accelerates the convergence speed due to the existence of the additional conjugate direction. Essentially, the optimal learning rate of each epoch is determined by the given inexact line search strategy. The presented model does not significantly increase the computational cost in dealing with real applications. Two benchmark simulations have been performed to illustrate the promising advantages of the proposed algorithm.