On the overfly algorithm in deep learning of neural networks

被引：2
|
作者：
Tsygvintsev, Alexei [1 ]
机构：
[1] Ecole Normale Super Lyon, UMPA, 46 Allee Italie, F-69364 Lyon 07, France
来源：
APPLIED MATHEMATICS AND COMPUTATION | 2019年 / 349卷
关键词：
Deep learning; Neural networks; Dynamical systems; Gradient descent; LOCAL MINIMA;
D O I：
10.1016/j.amc.2018.12.055
中图分类号：
O29 [应用数学];
学科分类号：
070104 ;
摘要：
In this paper, we investigate the supervised backpropagation training of multilayer neural networks from a dynamical systems point of view. We discuss some links with the qualitative theory of differential equations and introduce the overfly algorithm to tackle the local minima problem. Our approach is based on the existence of first integrals of the generalised gradient system with build-in dissipation. (C) 2019 Elsevier Inc. All rights reserved.
引用
收藏
页码：348 / 358
页数：11
相关论文
共 50 条
  • [31] Evolving Deep Neural Networks for Continuous Learning
    Atamanczuk, Bruna
    Karadas, Kurt Arve Skipenes
    Agrawal, Bikash
    Chakravorty, Antorweep
    FRONTIERS OF ARTIFICIAL INTELLIGENCE, ETHICS, AND MULTIDISCIPLINARY APPLICATIONS, FAIEMA 2023, 2024, : 3 - 16
  • [32] Evolutionary neural networks for deep learning: a review
    Ma, Yongjie
    Xie, Yirong
    INTERNATIONAL JOURNAL OF MACHINE LEARNING AND CYBERNETICS, 2022, 13 (10) : 3001 - 3018
  • [33] Learning deep neural networks for node classification
    Li, Bentian
    Pi, Dechang
    EXPERT SYSTEMS WITH APPLICATIONS, 2019, 137 : 324 - 334
  • [34] Learning to Play Othello With Deep Neural Networks
    Liskowski, Pawel
    Jaskowski, Wojciech
    Krawiec, Krzysztof
    IEEE TRANSACTIONS ON GAMES, 2018, 10 (04) : 354 - 364
  • [35] Deep Learning with Dense Random Neural Networks
    Gelenbe, Erol
    Yin, Yonghua
    MAN-MACHINE INTERACTIONS 5, ICMMI 2017, 2018, 659 : 3 - 18
  • [36] Can Deep Learning Only Be Neural Networks?
    Zhou, Zhi-Hua
    PROCEEDINGS OF THE 13TH INTERNATIONAL CONFERENCE ON WEB SEARCH AND DATA MINING (WSDM '20), 2020, : 6 - 6
  • [37] Deep Learning with Darwin: Evolutionary Synthesis of Deep Neural Networks
    Mohammad Javad Shafiee
    Akshaya Mishra
    Alexander Wong
    Neural Processing Letters, 2018, 48 : 603 - 613
  • [38] Deep Learning with Darwin: Evolutionary Synthesis of Deep Neural Networks
    Shafiee, Mohammad Javad
    Mishra, Akshaya
    Wong, Alexander
    NEURAL PROCESSING LETTERS, 2018, 48 (01) : 603 - 613
  • [39] Evolutionary neural networks for deep learning: a review
    Yongjie Ma
    Yirong Xie
    International Journal of Machine Learning and Cybernetics, 2022, 13 : 3001 - 3018
  • [40] DEEP LEARNING BASED METHOD FOR PRUNING DEEP NEURAL NETWORKS
    Li, Lianqiang
    Zhu, Jie
    Sun, Ming-Ting
    2019 IEEE INTERNATIONAL CONFERENCE ON MULTIMEDIA & EXPO WORKSHOPS (ICMEW), 2019, : 312 - 317
12345