Matrix-update for accelerated on-line learning in multilayer neural networks

被引：2
|
作者：
Bos, S [1 ]
机构：
[1] RIKEN, Lab Informat Synth, Brain Sci Inst, Wako, Saitama 35101, Japan
来源：
JOURNAL OF PHYSICS A-MATHEMATICAL AND GENERAL | 1998年 / 31卷 / 22期
关键词：
D O I：
10.1088/0305-4470/31/22/002
中图分类号：
O4 [物理学];
学科分类号：
0702 ;
摘要：
In this letter we re-examine the emergence of plateaus or symmetric phases in online learning of a committee machine. We propose a simple matrix-update in order to avoid the symmetric phase. Simulations show that the length of the plateaus can be considerably decreased. Annealing of the learning rate can then be applied much earlier, making on-line learning more effective.
引用
收藏
页码：L413 / L417
页数：5
相关论文
共 50 条
  • [1] Dynamics of on-line gradient descent learning for multilayer neural networks
    Saad, D
    Solla, SA
    ADVANCES IN NEURAL INFORMATION PROCESSING SYSTEMS 8: PROCEEDINGS OF THE 1995 CONFERENCE, 1996, 8 : 302 - 308
  • [2] Globally optimal parameters for on-line learning in multilayer neural networks
    Saad, D
    Rattray, M
    PHYSICAL REVIEW LETTERS, 1997, 79 (13) : 2578 - 2581
  • [3] Finite-size effects in on-line learning of multilayer neural networks
    Barber, D
    Saad, D
    Sollich, P
    EUROPHYSICS LETTERS, 1996, 34 (02): : 151 - 156
  • [4] On-line learning from restricted training sets in multilayer neural networks
    Coolen, ACC
    Saad, D
    Xiong, YS
    EUROPHYSICS LETTERS, 2000, 51 (06): : 691 - 697
  • [5] Dynamic multilayer neural networks for nonlinear system on-line identification
    Yu, W
    Poznyak, AS
    Sanchez, EN
    PROCEEDINGS OF THE 2000 IEEE INTERNATIONAL SYMPOSIUM ON INTELLIGENT CONTROL, 2000, : 25 - 30
  • [6] Multilayer neural networks:: an experimental evaluation of on-line training methods
    Martí, R
    El-Fallahi, A
    COMPUTERS & OPERATIONS RESEARCH, 2004, 31 (09) : 1491 - 1513
  • [7] A training data selection in on-line training for multilayer neural networks
    Hara, K
    Nakayama, K
    Kharaf, AAM
    IEEE WORLD CONGRESS ON COMPUTATIONAL INTELLIGENCE, 1998, : 2247 - 2252
  • [8] Adaptive back-propagation in on-line learning of multilayer networks
    West, AHL
    Saad, D
    ADVANCES IN NEURAL INFORMATION PROCESSING SYSTEMS 8: PROCEEDINGS OF THE 1995 CONFERENCE, 1996, 8 : 323 - 329
  • [9] On-line learning algorithms for locally recurrent neural networks
    Campolucci, P
    Uncini, A
    Piazza, F
    Rao, BD
    IEEE TRANSACTIONS ON NEURAL NETWORKS, 1999, 10 (02): : 253 - 271
  • [10] Harmonic neural networks for on-line learning vector quantisation
    Wang, JH
    Peng, CY
    Rau, JD
    IEE PROCEEDINGS-VISION IMAGE AND SIGNAL PROCESSING, 2000, 147 (05): : 485 - 492
12345