Performance Analysis of Feed Forward MLP with various Activation Functions for Handwritten Numerals Recognition

Aim of this paper is to analyze the performance of back-propagation feed-forward algorithm using various different activation functions for the neurons of hidden and output layers. For sample creation, 250 numerals were gathered form 35 people. After binarization, these numerals were clubbed together to form training patterns for the neural network. Network was trained to learn its behavior by adjusting the connection strengths at every iteration. The conjugate gradient descent of each presented training pattern was calculated to identify the minima on the error surface for each training pattern. Experiments were performed by selecting different combinations of two activation functions out of the three activation functions 'logsig', 'tansig' and 'purelin' for the neurons of the hidden and output layers and the results revealed that the percentage recognition accuracy of the neural network was observed to be optimum when 'tansig'-'tansig' combination of activation functions was used for neurons of hidden and output layers.