A SIMPLE AND EFFECTIVE METHOD FOR REMOVAL OF HIDDEN UNITS AND WEIGHTS

被引：55
作者：
HAGIWARA, M [1 ]
机构：
[1] KEIO UNIV, FAC SCI & TECHNOL, DEPT ELECT ENGN, KOHOKU KU, YOKOHAMA, KANAGAWA 223, JAPAN
来源：
NEUROCOMPUTING | 1994年 / 6卷 / 02期
关键词：
BACKPROPAGATION; UNITS REMOVAL; WEIGHTS REMOVAL; GENERALIZATION;
D O I：
10.1016/0925-2312(94)90055-8
中图分类号：
TP18 [人工智能理论];
学科分类号：
081104 ; 0812 ; 0835 ; 1405 ;
摘要：
The objective of this paper is to present a simple and effective method for removal of both hidden units and weights. In this paper, we propose two methods, the 'Consuming energy' method and the 'Weights power' method, and compare them with the conventional method. According to our computer simulations using the mirror symmetry problem, the Weights power method has shown the best performance with respect to size reduction (removal of units and weights), generalization performance, and the amount of computation required. For example, the number of hidden units reduced to about 40% of the initial state, and the number of weights reduced to less than a fourth of the initial state. In addition, generalization performance was improved more than 10%.
引用
收藏
页码：207 / 218
页数：12
相关论文
共 21 条
[1]  
CHAUVIN Y, 1989, ADV NEURAL INFORMATI, V1, P519
[2]   ANALYSIS OF HIDDEN UNITS IN A LAYERED NETWORK TRAINED TO CLASSIFY SONAR TARGETS [J].
GORMAN, RP ;
SEJNOWSKI, TJ .
NEURAL NETWORKS, 1988, 1 (01) :75-89
[3]  
HANSON SJ, 1989, ADV NEURAL INFORMATI, V1, P177
[4]  
HASSIBI B, 1992, CRCTR9235 RICOH CAL
[5]  
Hertz J.A, 1992, NIPS 91 P 4 INT C NE, P950
[6]  
HIGAWARA M, 1990, P INT JOINT C NEUR N, V1, P625
[7]  
Karnin E D, 1990, IEEE Trans Neural Netw, V1, P239, DOI 10.1109/72.80236
[8]  
KUNG SY, 1988, IEEE INT C NEURAL NE, V1, P363
[9]  
LeCun Y., 1990, PROC ADV NEURAL INFO, P598, DOI DOI 10.5555/109230.109298
[10]  
MATSUNAGA Y, 1991, T IEICE, V8, P1118
123