On the overfly algorithm in deep learning of neural networks

被引：2
作者：
Tsygvintsev, Alexei [1 ]
机构：
[1] Ecole Normale Super Lyon, UMPA, 46 Allee Italie, F-69364 Lyon 07, France
来源：
APPLIED MATHEMATICS AND COMPUTATION | 2019年 / 349卷
关键词：
Deep learning; Neural networks; Dynamical systems; Gradient descent; LOCAL MINIMA;
D O I：
10.1016/j.amc.2018.12.055
中图分类号：
O29 [应用数学];
学科分类号：
070104 ;
摘要：
In this paper, we investigate the supervised backpropagation training of multilayer neural networks from a dynamical systems point of view. We discuss some links with the qualitative theory of differential equations and introduce the overfly algorithm to tackle the local minima problem. Our approach is based on the existence of first integrals of the generalised gradient system with build-in dissipation. (C) 2019 Elsevier Inc. All rights reserved.
引用
收藏
页码：348 / 358
页数：11
相关论文
共 19 条
[1]   On the stable equilibrium points of gradient systems [J].
Absil, P-A. ;
Kurdyka, K. .
SYSTEMS & CONTROL LETTERS, 2006, 55 (07) :573-577
[2]  
[Anonymous], 2003, NONLINEAR PROGRAMMIN
[3]  
Atakulreka A., 2007, LECT NOTES COMPUTER, V4830
[4]  
Brierton J. L, 1997, PROC SPIE, V3066
[5]  
Burse K, 2011, COMM COM INF SC, V147, P67
[6]  
Busvelle E., 1994, APPL MATH, V22, P373
[7]  
CETIN BC, 1993, 1993 IEEE INTERNATIONAL CONFERENCE ON NEURAL NETWORKS, VOLS 1-3, P836, DOI 10.1109/ICNN.1993.298667
[8]   A modified back-propagation method to avoid false local minima [J].
Fukuoka, Y ;
Matsuki, H ;
Minamitani, H ;
Ishida, A .
NEURAL NETWORKS, 1998, 11 (06) :1059-1072
[9]  
Gallant S I, 1990, IEEE Trans Neural Netw, V1, P179, DOI 10.1109/72.80230
[10]   ON THE PROBLEM OF LOCAL MINIMA IN BACKPROPAGATION [J].
GORI, M ;
TESI, A .
IEEE TRANSACTIONS ON PATTERN ANALYSIS AND MACHINE INTELLIGENCE, 1992, 14 (01) :76-86
12