An efficient EM-based training algorithm for feedforward neural networks

被引：18
作者：
Ma, S
Ji, CY
Farmer, J
机构：
[1] Dept. of Elec., Comp. and Syst. Eng., Reusselaer Polytechnic Institute, Troy
来源：
NEURAL NETWORKS | 1997年 / 10卷 / 02期
关键词：
EM algorithm; feedforward networks; fast training; probability model of feedforward networks; hidden representations; linear weighted regression;
D O I：
10.1016/S0893-6080(96)00049-4
中图分类号：
TP18 [人工智能理论];
学科分类号：
081104 ; 0812 ; 0835 ; 1405 ;
摘要：
A fast training algorithm is developed for two-layer feedforward neural networks based on a probabilistic model for hidden representations and the EM algorithm. The algorithm decomposes training the original two-layer networks into training a set of single neurons. The individual neurons are then trained via a linear weighted regression algorithm. Significant improvement on training speed has been made using this algorithm for several bench-mark problems. (C) 1997 Elsevier Science Ltd. All Rights Reserved.
引用
收藏
页码：243 / 256
页数：14
相关论文
共 21 条
[1]   INFORMATION GEOMETRY OF BOLTZMANN MACHINES [J].
AMARI, S ;
KURATA, K ;
NAGAOKA, H .
IEEE TRANSACTIONS ON NEURAL NETWORKS, 1992, 3 (02) :260-271
[2]   THE EM ALGORITHM AND INFORMATION GEOMETRY IN NEURAL-NETWORK LEARNING [J].
AMARI, S .
NEURAL COMPUTATION, 1995, 7 (01) :13-18
[3]   UNIVERSAL APPROXIMATION BOUNDS FOR SUPERPOSITIONS OF A SIGMOIDAL FUNCTION [J].
BARRON, AR .
IEEE TRANSACTIONS ON INFORMATION THEORY, 1993, 39 (03) :930-945
[4]  
BREIMAN LE, 1993, NEURAL NETWORKS COMP
[5]   ALTERNATING MINIMIZATION AND BOLTZMANN MACHINE LEARNING [J].
BYRNE, W .
IEEE TRANSACTIONS ON NEURAL NETWORKS, 1992, 3 (04) :612-620
[6]  
Cybenko G., 1989, Mathematics of Control, Signals, and Systems, V2, P303, DOI 10.1007/BF02551274
[7]   MAXIMUM LIKELIHOOD FROM INCOMPLETE DATA VIA EM ALGORITHM [J].
DEMPSTER, AP ;
LAIRD, NM ;
RUBIN, DB .
JOURNAL OF THE ROYAL STATISTICAL SOCIETY SERIES B-METHODOLOGICAL, 1977, 39 (01) :1-38
[8]  
Fahlman S., 1990, ADV NEURAL INFORMATI, V2, P524
[9]   STOCHASTIC RELAXATION, GIBBS DISTRIBUTIONS, AND THE BAYESIAN RESTORATION OF IMAGES [J].
GEMAN, S ;
GEMAN, D .
IEEE TRANSACTIONS ON PATTERN ANALYSIS AND MACHINE INTELLIGENCE, 1984, 6 (06) :721-741
[10]   A UNIFIED FRAMEWORK FOR CONNECTIONIST SYSTEMS [J].
GOLDEN, RM .
BIOLOGICAL CYBERNETICS, 1988, 59 (02) :109-120
123