Comparison of Different Backpropagation Training Algorithms Using Robust M-Estimators Performance Functions

Artificial neural networks are one of the most popular and promising areas of artificial intelligence research. Training data containing outliers are often a problem for supervised neural networks learning algorithms that may not always come up with acceptable performance. Many robust learning algorithms have been proposed so far to improve the performance of neural networks in the presence of outliers. In this paper, we investigate the performance of four different backpropagation training algorithms, which are conjugate gradient with Fletcher - Reeves updates, conjugate gradient with Polak - Ribiere updates, resilient backpropagation, and conjugate gradient with Powell - peal restart. We compare their performance in terms of Root Mean Square Error as a merit function and the training speed in seconds. Examined neural networks trained by aforementioned backpropagation learning algorithms, which used the robust M-estimators performance functions instead of MSE one, in order to get robust learning in the presence of outliers. The study results show that Traincgf is the best algorithm in terms of mean square error, while the Traincgp is the best in terms of training speed.