On the overfly algorithm in deep learning of neural networks

被引：2

作者：

Tsygvintsev, Alexei ^{[1
]}

机构：

[1] Ecole Normale Super Lyon, UMPA, 46 Allee Italie, F-69364 Lyon 07, France

来源：

APPLIED MATHEMATICS AND COMPUTATION | 2019年 / 349卷

关键词：

Deep learning; Neural networks; Dynamical systems; Gradient descent; LOCAL MINIMA;

D O I：

10.1016/j.amc.2018.12.055

中图分类号：

O29 [应用数学];

学科分类号：

070104 ;

摘要：

In this paper, we investigate the supervised backpropagation training of multilayer neural networks from a dynamical systems point of view. We discuss some links with the qualitative theory of differential equations and introduce the overfly algorithm to tackle the local minima problem. Our approach is based on the existence of first integrals of the generalised gradient system with build-in dissipation. (C) 2019 Elsevier Inc. All rights reserved.

引用

页码：348 / 358

页数：11

共 50 条

[21] Introduction to Machine Learning, Neural Networks, and Deep Learning
Choi, Rene Y.
Coyner, Aaron S.
Kalpathy-Cramer, Jayashree
Chiang, Michael F.
Campbell, J. Peter
TRANSLATIONAL VISION SCIENCE & TECHNOLOGY, 2020, 9 (02):
[22] Multi-Adaptive Optimization for multi-task learning with deep neural networks
Hervella, alvaro S.
Rouco, Jose
Novo, Jorge
Ortega, Marcos
NEURAL NETWORKS, 2024, 170 : 254 - 265
[23] Design & Development of Deep Learning Algorithm with Convolutional Neural Networks for Breast Cancer Classification
Singh, Rajesh
Mishra, Ragini
2022 25TH INTERNATIONAL SYMPOSIUM ON WIRELESS PERSONAL MULTIMEDIA COMMUNICATIONS (WPMC), 2022,
[24] Global learning of neural networks by using hybrid optimization algorithm
Cho, Yong-Hyun
Hong, Seong-Jun
PROCEEDINGS OF THE INTERNATIONAL CONFERENCE ON INTELLIGENT SYSTEMS AND KNOWLEDGE ENGINEERING (ISKE 2007), 2007,
[25] AdamRAG: Adaptive Algorithm with Ravine Method for Training Deep Neural Networks
Yifan Zhang
Di Zhao
Hongyi Li
Chengwei Pan
Neural Processing Letters, 57 (3)
[26] Experiential Robot Learning with Deep Neural Networks
Aly, Ahmed A.
Dugan, Joanne Bechta
2017 THE SEVENTH JOINT IEEE INTERNATIONAL CONFERENCE ON DEVELOPMENT AND LEARNING AND EPIGENETIC ROBOTICS (ICDL-EPIROB), 2017, : 356 - 361
[27] Robust learning of parsimonious deep neural networks
Guenter, Valentin Frank Ingmar
Sideris, Athanasios
NEUROCOMPUTING, 2024, 566
[28] Accretionary Learning With Deep Neural Networks With Applications
Wei, Xinyu
Juang, Biing-Hwang
Wang, Ouya
Zhou, Shenglong
Li, Geoffrey Ye
IEEE TRANSACTIONS ON COGNITIVE COMMUNICATIONS AND NETWORKING, 2024, 10 (02) : 660 - 673
[29] Representational Distance Learning for Deep Neural Networks
McClure, Patrick
Kriegeskorte, Nikolaus
FRONTIERS IN COMPUTATIONAL NEUROSCIENCE, 2016, 10
[30] Learning hidden chemistry with deep neural networks
Nguyen, Tien-Cuong
Nguyen, Van-Quyen
Ngo, Van-Linh
Than, Quang-Khoat
Pham, Tien-Lam
COMPUTATIONAL MATERIALS SCIENCE, 2021, 200

← 1 2 3 4 5 →