On the overfly algorithm in deep learning of neural networks

被引：2
|
作者：
Tsygvintsev, Alexei [1 ]
机构：
[1] Ecole Normale Super Lyon, UMPA, 46 Allee Italie, F-69364 Lyon 07, France
来源：
APPLIED MATHEMATICS AND COMPUTATION | 2019年 / 349卷
关键词：
Deep learning; Neural networks; Dynamical systems; Gradient descent; LOCAL MINIMA;
D O I：
10.1016/j.amc.2018.12.055
中图分类号：
O29 [应用数学];
学科分类号：
070104 ;
摘要：
In this paper, we investigate the supervised backpropagation training of multilayer neural networks from a dynamical systems point of view. We discuss some links with the qualitative theory of differential equations and introduce the overfly algorithm to tackle the local minima problem. Our approach is based on the existence of first integrals of the generalised gradient system with build-in dissipation. (C) 2019 Elsevier Inc. All rights reserved.
引用
收藏
页码：348 / 358
页数：11
相关论文
共 50 条
  • [1] Deep learning in spiking neural networks
    Tavanaei, Amirhossein
    Ghodrati, Masoud
    Kheradpisheh, Saeed Reza
    Masquelier, Timothee
    Maida, Anthony
    NEURAL NETWORKS, 2019, 111 : 47 - 63
  • [2] Development of an Algorithm for Multicriteria Optimization of Deep Learning Neural Networks
    Alexandrov I.A.
    Kirichek A.V.
    Kuklin V.Z.
    Chervyakov L.M.
    HighTech and Innovation Journal, 2023, 4 (01): : 157 - 173
  • [3] ABOUT AN ALGORITHM FOR CONSISTENT WEIGHTS INITIALIZATION OF DEEP NEURAL NETWORKS AND NEURAL NETWORKS ENSEMBLE LEARNING
    Drokin, I. S.
    VESTNIK SANKT-PETERBURGSKOGO UNIVERSITETA SERIYA 10 PRIKLADNAYA MATEMATIKA INFORMATIKA PROTSESSY UPRAVLENIYA, 2016, 12 (04): : 66 - 74
  • [4] Deep Learning for Epidemiologists: An Introduction to Neural Networks
    Serghiou, Stylianos
    Rough, Kathryn
    AMERICAN JOURNAL OF EPIDEMIOLOGY, 2023, 192 (11) : 1904 - 1916
  • [5] PHYLOGENETIC REPLAY LEARNING IN DEEP NEURAL NETWORKS
    Glafkides, Jean-Patrice
    Sher, Gene, I
    Akdag, Herman
    JORDANIAN JOURNAL OF COMPUTERS AND INFORMATION TECHNOLOGY, 2022, 8 (03): : 218 - 231
  • [6] Parameter incremental learning algorithm for neural networks
    Wan, Sheng
    Banta, Larry E.
    IEEE TRANSACTIONS ON NEURAL NETWORKS, 2006, 17 (06): : 1424 - 1438
  • [7] A Novel Learning Algorithm to Optimize Deep Neural Networks: Evolved Gradient Direction Optimizer (EVGO)
    Karabayir, Ibrahim
    Akbilgic, Oguz
    Tas, Nihat
    IEEE TRANSACTIONS ON NEURAL NETWORKS AND LEARNING SYSTEMS, 2021, 32 (02) : 685 - 694
  • [8] The Whale Optimization Algorithm Approach for Deep Neural Networks
    Brodzicki, Andrzej
    Piekarski, Michal
    Jaworek-Korjakowska, Joanna
    SENSORS, 2021, 21 (23)
  • [9] DeepCABAC: A Universal Compression Algorithm for Deep Neural Networks
    Wiedemann, Simon
    Kirchhoffer, Heiner
    Matlage, Stefan
    Haase, Paul
    Marban, Arturo
    Marinc, Talmaj
    Neumann, David
    Nguyen, Tung
    Schwarz, Heiko
    Wiegand, Thomas
    Marpe, Detlev
    Samek, Wojciech
    IEEE JOURNAL OF SELECTED TOPICS IN SIGNAL PROCESSING, 2020, 14 (04) : 700 - 714
  • [10] QLP: Deep Q-Learning for Pruning Deep Neural Networks
    Camci, Efe
    Gupta, Manas
    Wu, Min
    Lin, Jie
    IEEE TRANSACTIONS ON CIRCUITS AND SYSTEMS FOR VIDEO TECHNOLOGY, 2022, 32 (10) : 6488 - 6501
12345