Boundedness and Convergence of Online Gradient Method with Penalty for Linear Output Feedforward Neural Networks

被引：0
|
作者：
Huisheng Zhang
Wei Wu
机构：
[1] Dalian University of Technology,Department of Applied Mathematics
[2] Dalian Maritime University,Department of Mathematics
来源：
Neural Processing Letters | 2009年 / 29卷
关键词：
Feedforward neural networks; Linear output; Online gradient method; Penalty; Boundedness; Convergence;
D O I：
暂无
中图分类号：
学科分类号：
摘要：
This paper investigates an online gradient method with penalty for training feedforward neural networks with linear output. A usual penalty is considered, which is a term proportional to the norm of the weights. The main contribution of this paper is to theoretically prove the boundedness of the weights in the network training process. This boundedness is then used to prove an almost sure convergence of the algorithm to the zero set of the gradient of the error function.
引用
收藏
页码：205 / 212
页数：7
相关论文
共 50 条
  • [1] Boundedness and Convergence of Online Gradient Method with Penalty for Linear Output Feedforward Neural Networks
    Zhang, Huisheng
    Wu, Wei
    NEURAL PROCESSING LETTERS, 2009, 29 (03) : 205 - 212
  • [2] Boundedness and Convergence of Online Gradient Method With Penalty for Feedforward Neural Networks
    Zhang, Huisheng
    Wu, Wei
    Liu, Fei
    Yao, Mingchen
    IEEE TRANSACTIONS ON NEURAL NETWORKS, 2009, 20 (06): : 1050 - 1054
  • [3] Boundedness and convergence of online gradient method with penalty and momentum
    Shao, Hongmei
    Zheng, Gaofeng
    NEUROCOMPUTING, 2011, 74 (05) : 765 - 770
  • [4] Boundedness of a batch gradient method with penalty for feedforward neural networks
    Zhang, Huisheng
    Wu, Wei
    Yao, Mingchen
    APPLIED MATHEMATICS FOR SCIENCE AND ENGINEERING, 2007, : 175 - +
  • [5] Analysis of Boundedness and Convergence of Online Gradient Method for Two-Layer Feedforward Neural Networks
    Xu, Lu
    Chen, Jinshu
    Huang, Defeng
    Lu, Jianhua
    Fang, Licai
    IEEE TRANSACTIONS ON NEURAL NETWORKS AND LEARNING SYSTEMS, 2013, 24 (08) : 1327 - 1338
  • [6] Boundedness and convergence of batch back-propagation algorithm with penalty for feedforward neural networks
    Zhang, Huisheng
    Wu, Wei
    Yao, Mingchen
    NEUROCOMPUTING, 2012, 89 : 141 - 146
  • [7] Convergence of online gradient method for feedforward neural networks with smoothing L1/2 regularization penalty
    Fan, Qinwei
    Zurada, Jacek M.
    Wu, Wei
    NEUROCOMPUTING, 2014, 131 : 208 - 216
  • [8] CONVERGENCE OF ONLINE GRADIENT METHOD WITH A PENALTY TERM FOR FEEDFORWARD NEURAL NETWORKS WITH STOCHASTIC INPUTS
    邵红梅
    吴微
    李峰
    NumericalMathematicsAJournalofChineseUniversities(EnglishSeries), 2005, (01) : 87 - 96
  • [9] Convergence of Online Gradient Method with Penalty for BP Neural Networks
    Shao Hong-mei 1
    Communications in Mathematical Research, 2010, 26 (01) : 67 - 75
  • [10] Convergence of an online gradient method for feedforward neural networks with stochastic inputs
    Li, ZX
    Wu, W
    Tian, YL
    JOURNAL OF COMPUTATIONAL AND APPLIED MATHEMATICS, 2004, 163 (01) : 165 - 176
12345