A pruning algorithm of neural networks using impact factor regularization

In general, small-sized networks, even though they show good generalization performance, tend to fail to learn the training data within a given error bound, whereas largesized networks learn easily the training data but yield poor generalization. In this paper, a pruning algorithm of neural networks using impact factor regularization is described to train network without overfitting and to achieve a smallsized network. In order to achieve this goal, an automatic determination method of the regularization parameter and an extended Levenberg-Marquardt algorithm are developed as learning algorithms of neural networks. We tested the proposed method on four regression problems and the simulation results showed our algorithm is effective in regression.