Convergence of a Batch Gradient Algorithm with Adaptive Momentum for Neural Networks

被引：0
作者：
Hongmei Shao
Dongpo Xu
Gaofeng Zheng
机构：
[1] China University of Petroleum,School of Math. and Comput. Science
[2] College of Science,undefined
[3] Harbin Engineering University,undefined
[4] Platform Search Group,undefined
[5] Rakuten Inc.,undefined
来源：
Neural Processing Letters | 2011年 / 34卷
关键词：
Neural network; Gradient algorithm; Adaptive momentum; Convergence;
D O I：
暂无
中图分类号：
学科分类号：
摘要：
In this paper, a batch gradient algorithm with adaptive momentum is considered and a convergence theorem is presented when it is used for two-layer feedforward neural networks training. Simple but necessary sufficient conditions are offered to guarantee both weak and strong convergence. Compared with existing general requirements, we do not restrict the error function to be quadratic or uniformly convex. A numerical example is supplied to illustrate the performance of the algorithm and support our theoretical finding.
引用
收藏
页码：221 / 228
页数：7
相关论文
共 31 条
  • [1] Meybodi MR(2002)A note on learning automata-based schemes for adaptation of BP parameters Neurocomputing 48 957-974
  • [2] Beigy H(1992)Accelerated training of backpropagation networks by using adaptive momentum step IEE Electron Lett 28 377-379
  • [3] Qiu G(2002)Improved backpropagation learning in nerural networks with windowed momentum Int J Neural Syst 12 303-318
  • [4] Varley MR(1987)An adaptive training algorithm for backpropagation networks Comput Speech Lang 2 205-218
  • [5] Terrell TJ(1993)A new acceleration technique for the backpropagation algorithm IEEE Int Conf Neural Netw 3 1157-1161
  • [6] Istook E(2002)A backpropagation algorithm with adaptive learning rate and momentum coefficient IEEE Int Conf Neural Netw 2 1218-1223
  • [7] Martinez T(2009)A new BP algorithm with adaptive momentum for FNNs training WRI Glob Congr Intell Syst 4 16-20
  • [8] Chan LW(2004)Steepest descent with momentum for quadratic functions is a version of the conjugate gradient method Neural Netw 17 65-71
  • [9] Fallside F(2002)Stability of steepest descent with momentum for quadratic functions IEEE Trans Neural Netw 13 752-756
  • [10] Yu X(2006)Convergence of gradient method with momentum for two-layer feedforward neural networks IEEE Trans Neural Netw 17 522-525
1234