Computing and Analyzing the Sensitivity of MLP Due to the Errors of the i.i.d. Inputs and Weights Based on CLT

被引:11
作者
Yang, Sheng-Sung [1 ]
Ho, Chia-Lu [2 ]
Siu, Sammy [3 ]
机构
[1] Natl Cent Univ, Inst Elect Engn, Chungli 32054, Taiwan
[2] Natl Cent Univ, Inst Commun Engn, Chungli 32054, Taiwan
[3] Chunghwa Telecom Co, Telecommun Labs, Tao Yuan 326, Taiwan
来源
IEEE TRANSACTIONS ON NEURAL NETWORKS | 2010年 / 21卷 / 12期
关键词
Central limit theorem; multilayer perceptron; neural networks; sensitivity analysis; MULTILAYER PERCEPTRON; PERTURBATION; NETWORKS;
D O I
10.1109/TNN.2010.2077681
中图分类号
TP18 [人工智能理论];
学科分类号
081104 ; 0812 ; 0835 ; 1405 ;
摘要
In this paper, we propose an algorithm based on the central limit theorem to compute the sensitivity of the multilayer perceptron (MLP) due to the errors of the inputs and weights. For simplicity and practicality, all inputs and weights studied here are independently identically distributed (i.i.d.). The theoretical results derived from the proposed algorithm show that the sensitivity of the MLP is affected by the number of layers and the number of neurons adopted in each layer. To prove the reliability of the proposed algorithm, some experimental results of the sensitivity are also presented, and they match the theoretical ones. The good agreement between the theoretical results and the experimental results verifies the reliability and feasibility of the proposed algorithm. Furthermore, the proposed algorithm can also be applied to compute precisely the sensitivity of the MLP with any available activation functions and any types of i.i.d. inputs and weights.
引用
收藏
页码:1882 / 1891
页数:10
相关论文
共 18 条
[1]  
[Anonymous], Probability, Random Variables and Stochastic Processes
[2]   Improving the tolerance of multilayer perceptrons by minimizing the statistical sensitivity to weight deviations [J].
Bernier, JL ;
Ortega, J ;
Rojas, I ;
Prieto, A .
NEUROCOMPUTING, 2000, 31 (1-4) :87-103
[3]   Sensitivity analysis of neocognitron [J].
Cheng, AY ;
Yeung, DS .
IEEE TRANSACTIONS ON SYSTEMS MAN AND CYBERNETICS PART C-APPLICATIONS AND REVIEWS, 1999, 29 (02) :238-249
[4]   A new pruning heuristic based on variance analysis of sensitivity information [J].
Engelbrecht, AP .
IEEE TRANSACTIONS ON NEURAL NETWORKS, 2001, 12 (06) :1386-1399
[5]  
ENGELBRECHT AP, 1996, P IEEE INT C NEUR NE, V2, P1274
[6]  
ENGELBRECHT AP, 1999, P IEEE INT JOINT C N, V3, P1829
[7]  
FU L, 1993, P IEEE INT C NEUR NE, V1, P215
[8]  
HASHEM S, 1992, P 1992 INT JOINT C N, V1, P419
[9]   SENSITIVITY ANALYSIS OF MULTILAYER PERCEPTRON WITH DIFFERENTIABLE ACTIVATION FUNCTIONS [J].
JIN, YC ;
CHOI, CH .
IEEE TRANSACTIONS ON NEURAL NETWORKS, 1992, 3 (01) :101-107
[10]   SENSITIVITY ANALYSIS OF SINGLE HIDDEN-LAYER NEURAL NETWORKS WITH THRESHOLD FUNCTIONS [J].
OH, SH ;
LEE, YJ .
IEEE TRANSACTIONS ON NEURAL NETWORKS, 1995, 6 (04) :1005-1007