Complexity issues in natural gradient descent method for training multilayer perceptrons

被引：54

作者：

Yang, HH ^{[1
]}

Amari, S

机构：

[1] Oregon Grad Inst, Dept Comp Sci, Portland, OR 97291 USA

[2] RIKEN, Brain Sci Inst, Lab Informat Synth, Wako, Saitama 35101, Japan

来源：

NEURAL COMPUTATION | 1998年 / 10卷 / 08期

关键词：

D O I：

10.1162/089976698300017007

中图分类号：

TP18 [人工智能理论];

学科分类号：

081104 ; 0812 ; 0835 ; 1405 ;

摘要：

The natural gradient descent method is applied to train an n-m-1 multilayer perceptron. Based on an efficient scheme to represent the Fisher information matrix for an n-m-1 stochastic multilayer perceptron, a new algorithm is proposed to calculate the natural gradient without inverting the Fisher information matrix explicitly. When the input dimension n is much larger than the number of hidden neurons m, the time complexity of computing the natural gradient is O(n).

引用

页码：2137 / 2157

页数：21

共 9 条

[1] Natural gradient works efficiently in learning [J].

Amari, S .

NEURAL COMPUTATION, 1998, 10 (02) :251-276

[2]

Amari S, 1997, ADV NEUR IN, V9, P127

[3]

ORR GB, 1997, ADV NEURAL INFORMATI, V9

[4] ONLINE LEARNING IN SOFT COMMITTEE MACHINES [J].

SAAD, D ;

SOLLA, SA .

PHYSICAL REVIEW E, 1995, 52 (04) :4225-4243

[5]

Stewart GW., 1973, INTRO MATRIX COMPUTA

[6] Statistical inference: learning in artificial neural networks [J].

Yang, HH ;

Murata, N ;

Amari, S .

TRENDS IN COGNITIVE SCIENCES, 1998, 2 (01) :4-10

[7]

YANG HH, 1997, ICONIP 97 P NZ

[8]

YANG HH, 1997, UNPUB NATURAL GRADIE

[9]

YANG HH, 1998, ADV NEURAL INFORMATI, V10

← 1 →