On the overfly algorithm in deep learning of neural networks

被引：2
|
作者：
Tsygvintsev, Alexei [1 ]
机构：
[1] Ecole Normale Super Lyon, UMPA, 46 Allee Italie, F-69364 Lyon 07, France
来源：
APPLIED MATHEMATICS AND COMPUTATION | 2019年 / 349卷
关键词：
Deep learning; Neural networks; Dynamical systems; Gradient descent; LOCAL MINIMA;
D O I：
10.1016/j.amc.2018.12.055
中图分类号：
O29 [应用数学];
学科分类号：
070104 ;
摘要：
In this paper, we investigate the supervised backpropagation training of multilayer neural networks from a dynamical systems point of view. We discuss some links with the qualitative theory of differential equations and introduce the overfly algorithm to tackle the local minima problem. Our approach is based on the existence of first integrals of the generalised gradient system with build-in dissipation. (C) 2019 Elsevier Inc. All rights reserved.
引用
收藏
页码：348 / 358
页数：11
相关论文
共 50 条
  • [41] Automatic Modulation Classification: Convolutional Deep Learning Neural Networks Approaches
    Hussein, Hany S.
    Essai Ali, Mohamed Hassan
    Ismeil, Mohammed
    Shaaban, Mohamed N.
    Mohamed, Mona Lotfy
    Atallah, Hany A.
    IEEE ACCESS, 2023, 11 : 98695 - 98705
  • [42] Neural networks with robust backpropagation learning algorithm
    Walczak, B
    ANALYTICA CHIMICA ACTA, 1996, 322 (1-2) : 21 - 29
  • [43] Constraints on Hyper-parameters in Deep Learning Convolutional Neural Networks
    Al-Saggaf, Ubaid M.
    Botalb, Abdelaziz
    Faisal, Muhammad
    Moinuddin, Muhammad
    Alsaggaf, Abdulrahman U.
    Alfakeh, Sulhi Ali
    INTERNATIONAL JOURNAL OF ADVANCED COMPUTER SCIENCE AND APPLICATIONS, 2022, 13 (11) : 439 - 449
  • [44] Learning fashion compatibility across categories with deep multimodal neural networks
    Sun, Guang-Lu
    He, Jun-Yan
    Wu, Xiao
    Zhao, Bo
    Peng, Qiang
    NEUROCOMPUTING, 2020, 395 (237-246) : 237 - 246
  • [45] Deep Learning Neural Networks Optimization using Hardware Cost Penalty
    Doshi, Rohan
    Hung, Kwok-Wai
    Liang, Luhong
    Chiu, King-Hung
    2016 IEEE INTERNATIONAL SYMPOSIUM ON CIRCUITS AND SYSTEMS (ISCAS), 2016, : 1954 - 1957
  • [46] NeuroMask: Explaining Predictions of Deep Neural Networks through Mask Learning
    Alzantot, Moustafa
    Widdicombe, Amy
    Julier, Simon
    Srivastava, Mani
    2019 IEEE INTERNATIONAL CONFERENCE ON SMART COMPUTING (SMARTCOMP 2019), 2019, : 81 - 86
  • [47] A Deep Learning Framework for Automated Transfer Learning of Neural Networks
    Balaiah, Thanasekhar
    Jeyadoss, Timothy Jones Thomas
    Thirumurugan, Sainee
    Ravi, Rahul Chander
    2019 11TH INTERNATIONAL CONFERENCE ON ADVANCED COMPUTING (ICOAC 2019), 2019, : 428 - 432
  • [48] Adaptive Learning Rate for Unsupervised Learning of Deep Neural Networks
    Golovko, Vladimir
    Mikhno, Egor
    Kroschanka, Aliaksandr
    Chodyka, Marta
    Lichograj, Piotr
    2023 INTERNATIONAL JOINT CONFERENCE ON NEURAL NETWORKS, IJCNN, 2023,
  • [49] Supervised Learning Algorithm Based on Spike Train Inner Product for Deep Spiking Neural Networks
    Lin, Xianghong
    Zhang, Zhen
    Zheng, Donghao
    BRAIN SCIENCES, 2023, 13 (02)
  • [50] An enhanced deep neural networks algorithm for hand segmentation
    Babatope, Eyitomilayo Yemisi
    Garcia-Vazquez, Mireya Sarai
    Ramirez-Acosta, Alejandro Alvaro
    OPTICS AND PHOTONICS FOR INFORMATION PROCESSING XVIII, 2024, 13136
12345