On the overfly algorithm in deep learning of neural networks

被引：2
|
作者：
Tsygvintsev, Alexei [1 ]
机构：
[1] Ecole Normale Super Lyon, UMPA, 46 Allee Italie, F-69364 Lyon 07, France
来源：
APPLIED MATHEMATICS AND COMPUTATION | 2019年 / 349卷
关键词：
Deep learning; Neural networks; Dynamical systems; Gradient descent; LOCAL MINIMA;
D O I：
10.1016/j.amc.2018.12.055
中图分类号：
O29 [应用数学];
学科分类号：
070104 ;
摘要：
In this paper, we investigate the supervised backpropagation training of multilayer neural networks from a dynamical systems point of view. We discuss some links with the qualitative theory of differential equations and introduce the overfly algorithm to tackle the local minima problem. Our approach is based on the existence of first integrals of the generalised gradient system with build-in dissipation. (C) 2019 Elsevier Inc. All rights reserved.
引用
收藏
页码：348 / 358
页数：11
相关论文
共 50 条
  • [11] Padding Module: Learning the Padding in Deep Neural Networks
    Alrasheedi, Fahad
    Zhong, Xin
    Huang, Pei-Chi
    IEEE ACCESS, 2023, 11 : 7348 - 7357
  • [12] IMPROVED MUSIC FEATURE LEARNING WITH DEEP NEURAL NETWORKS
    Sigtia, Siddharth
    Dixon, Simon
    2014 IEEE INTERNATIONAL CONFERENCE ON ACOUSTICS, SPEECH AND SIGNAL PROCESSING (ICASSP), 2014,
  • [13] Cryptocurrency forecasting with deep learning chaotic neural networks
    Lahmiri, Salim
    Bekiros, Stelios
    CHAOS SOLITONS & FRACTALS, 2019, 118 : 35 - 40
  • [14] Deep learning in neural networks: An overview
    Schmidhuber, Juergen
    NEURAL NETWORKS, 2015, 61 : 85 - 117
  • [15] Fast learning in Deep Neural Networks
    Chandra, B.
    Sharma, Rajesh K.
    NEUROCOMPUTING, 2016, 171 : 1205 - 1215
  • [16] Deep associative learning for neural networks
    Liu, Jia
    Zhang, Wenhua
    Liu, Fang
    Xiao, Liang
    NEUROCOMPUTING, 2021, 443 (443) : 222 - 234
  • [17] Deep Learning with Random Neural Networks
    Gelenbe, Erol
    Yin, Yongha
    PROCEEDINGS OF SAI INTELLIGENT SYSTEMS CONFERENCE (INTELLISYS) 2016, VOL 2, 2018, 16 : 450 - 462
  • [18] An evolutionary building algorithm for Deep Neural Networks
    Zemouri, Ryad
    2017 12TH INTERNATIONAL WORKSHOP ON SELF-ORGANIZING MAPS AND LEARNING VECTOR QUANTIZATION, CLUSTERING AND DATA VISUALIZATION (WSOM), 2017, : 21 - 27
  • [19] Iterative surrogate model optimization (ISMO): An active learning algorithm for PDE constrained optimization with deep neural networks
    Lye, Kjetil O.
    Mishra, Siddhartha
    Ray, Deep
    Chandrashekar, Praveen
    COMPUTER METHODS IN APPLIED MECHANICS AND ENGINEERING, 2021, 374
  • [20] Deep Learning based on Image Recognition Convolutional Neural Networks
    Alamri, Salah
    INTERNATIONAL JOURNAL OF COMPUTER SCIENCE AND NETWORK SECURITY, 2022, 22 (04): : 559 - 566
12345