Deterministic convergence of complex mini-batch gradient learning algorithm for fully complex-valued neural networks

被引:12
作者
Zhang, Huisheng [1 ]
Zhang, Ying [1 ]
Zhu, Shuai [1 ]
Xu, Dongpo [2 ]
机构
[1] Dalian Maritime Univ, Sch Sci, Dalian 116026, Peoples R China
[2] Northeast Normal Univ, Sch Math & Stat, Changchun 130024, Peoples R China
基金
中国国家自然科学基金;
关键词
Fully complex-valued neural networks; Mini-batch gradient algorithm; Convergence; Wirtinger calculus; BACKPROPAGATION ALGORITHM; PERFORMANCE BOUNDS; MOMENTUM; BOUNDEDNESS; ESTIMATORS; LMS;
D O I
10.1016/j.neucom.2020.04.114
中图分类号
TP18 [人工智能理论];
学科分类号
081104 ; 0812 ; 0835 ; 1405 ;
摘要
This paper investigates the fully complex mini-batch gradient algorithm for training complex-valued neural networks. Mini-batch gradient method has been widely used in neural network training, however, its convergence analysis is usually restricted to real-valued neural networks and of probability nature. By introducing a new Taylor mean value theorem for analytic functions, in this paper we establish determin-istic convergence results for the fully complex mini-batch gradient algorithm under mild conditions. The deterministic convergence here means that the algorithm will deterministically converge, and both the weak convergence and strong convergence will be proved. Benefited from the newly introduced mean value theorem, our results are of global nature in that they are valid for arbitrarily given initial values of the weights. The theoretical findings are validated with a simulation example. (C) 2020 Elsevier B.V. All rights reserved.
引用
收藏
页码:185 / 193
页数:9
相关论文
共 44 条
  • [1] Complex ICA using nonlinear functions
    Adali, Tuelay
    Li, Hualiang
    Novey, Mike
    Cardoso, Jean-Francois
    [J]. IEEE TRANSACTIONS ON SIGNAL PROCESSING, 2008, 56 (09) : 4536 - 4544
  • [2] Gradient convergence in gradient methods with errors
    Bertsekas, DP
    Tsitsiklis, JN
    [J]. SIAM JOURNAL ON OPTIMIZATION, 2000, 10 (03) : 627 - 642
  • [3] The Generalized Complex Kernel Least-Mean-Square Algorithm
    Boloix-Tortosa, Rafael
    Jose Murillo-Fuentes, Juan
    Tsaftaris, Sotirios A.
    [J]. IEEE TRANSACTIONS ON SIGNAL PROCESSING, 2019, 67 (20) : 5213 - 5222
  • [4] Brandwood D. H., 1983, IEE Proceedings H (Microwaves, Optics and Antennas), V130, P11, DOI 10.1049/ip-h-1.1983.0004
  • [5] Deep, Big, Simple Neural Nets for Handwritten Digit Recognition
    Ciresan, Dan Claudiu
    Meier, Ueli
    Gambardella, Luca Maria
    Schmidhuber, Juergen
    [J]. NEURAL COMPUTATION, 2010, 22 (12) : 3207 - 3220
  • [6] Parameter convergence and learning curves for neural networks
    Fine, TL
    Mukherjee, S
    [J]. NEURAL COMPUTATION, 1999, 11 (03) : 747 - 769
  • [7] COMPLEX-DOMAIN BACKPROPAGATION
    GEORGIOU, GM
    KOUTSOUGERAS, C
    [J]. IEEE TRANSACTIONS ON CIRCUITS AND SYSTEMS II-ANALOG AND DIGITAL SIGNAL PROCESSING, 1992, 39 (05): : 330 - 334
  • [8] Complex-valued forecasting of wind profile
    Goh, S. L.
    Chen, M.
    Popovic, D. H.
    Aihara, K.
    Obradovic, D.
    Mandic, D. P.
    [J]. RENEWABLE ENERGY, 2006, 31 (11) : 1733 - 1750
  • [9] A complex-valued RTRL algorithm for recurrent neural networks
    Goh, SL
    Mandic, DP
    [J]. NEURAL COMPUTATION, 2004, 16 (12) : 2699 - 2713
  • [10] Stochastic gradient-adaptive complex-valued nonlinear neural adaptive filters with a gradient-adaptive step size
    Goh, Su Lee
    Mandic, Danilo P.
    [J]. IEEE TRANSACTIONS ON NEURAL NETWORKS, 2007, 18 (05): : 1511 - 1516