A memory optimal BFGS neural network training algorithm

被引：8

作者：

McLoone, SF ^{[1
]}

Asirvadam, VS ^{[1
]}

Irwin, GW ^{[1
]}

机构：

[1] Queens Univ Belfast, Intelligent Syst & Control Grp, Sch Elect & Elect Engn, Belfast BT9 5AH, Antrim, North Ireland

来源：

PROCEEDING OF THE 2002 INTERNATIONAL JOINT CONFERENCE ON NEURAL NETWORKS, VOLS 1-3 | 2002年

关键词：

D O I：

10.1109/IJCNN.2002.1005525

中图分类号：

TP18 [人工智能理论];

学科分类号：

081104 ; 0812 ; 0835 ; 1405 ;

摘要：

This paper considers the implementation of a novel memory optimal neural network training algorithm which maximises performance in relation to available memory. Mathematically it is similar to full memory (FM) BFGS (Broyden, Fletcher, Goldfarb and Shanno) training when there are no constraints on memory and to the variable memory (VM) BFGS when memory is limited. However, it requires less computations per iteration than VM and uses a much better strategy for discarding old curvature information when memory is limited.

引用

页码：513 / 518

页数：4

共 17 条

[1] [Anonymous], 1986, PARALLEL DISTRIBUTED
[2] 1ST-ORDER AND 2ND-ORDER METHODS FOR LEARNING - BETWEEN STEEPEST DESCENT AND NEWTON METHOD
BATTITI, R
[J]. NEURAL COMPUTATION, 1992, 4 (02) : 141 - 166
[3] QN-LIKE VARIABLE STORAGE CONJUGATE GRADIENTS
BUCKLEY, A
LENIR, A
[J]. MATHEMATICAL PROGRAMMING, 1983, 27 (02) : 155 - 175
[4] TRAINING FEEDFORWARD NETWORKS WITH THE MARQUARDT ALGORITHM
HAGAN, MT
MENHAJ, MB
[J]. IEEE TRANSACTIONS ON NEURAL NETWORKS, 1994, 5 (06): : 989 - 993
[5] APPROXIMATELY TIME-OPTIMAL FUZZY CONTROL OF A 2-TANK SYSTEM
HECKENTHALER, T
ENGELL, S
[J]. IEEE CONTROL SYSTEMS MAGAZINE, 1994, 14 (03): : 24 - 30
[6] ON THE LIMITED MEMORY BFGS METHOD FOR LARGE-SCALE OPTIMIZATION
LIU, DC
NOCEDAL, J
[J]. MATHEMATICAL PROGRAMMING, 1989, 45 (03) : 503 - 528
[7] Fast parallel off-line training of multilayer perceptrons
McLoone, S
Irwin, GW
[J]. IEEE TRANSACTIONS ON NEURAL NETWORKS, 1997, 8 (03): : 646 - 653
[8] A variable memory Quasi-Newton training algorithm
McLoone, S
Irwin, G
[J]. NEURAL PROCESSING LETTERS, 1999, 9 (01) : 77 - 89
[9] MCLOONE S, 1995, NEURAL NETWORK ENG D, P179
[10] MCLOONE S, 2000, P 2 INT ICSC S NEUR, P185

← 1 2 →