The generalized sigmoid activation function: Competitive supervised learning

Multilayer perceptron (MLP) networks trained using backpropagation are perhaps the most commonly used neural network model. Central to the MLP model is the use of neurons with nonlinear and differentiable activation functions. The most commonly used activation function is a sigmoidal function, and frequently all neurons in an MLP network employ the same activation function. In this paper, we introduce the notion of the generalized sigmoid as an activation function for neurons in the output layer of an MLP network. The enhancements afforded by the use of the generalized sigmoid are analyzed and demonstrated in the context of some well-known classification problems. (C) Elsevier Science Inc. 1997.