Extreme Learning Machine based on Rectified Nonlinear Units

Traditional Extreme Learning Machine (ELM) networks generally used S-shaped activation function, such as Sigmoid function and Tangent function. However, the problems of slow convergence speed and over-saturation exist. In order to solve the above problems and improve the performance of ELM algorithm, the method of Rectified Non-Linear Units (ReNLUs), combining rectified linear units (ReLUs) with Softplus function method, was proposed. And the ReLUs has the ability of sparse expression and the Softplus possesses smooth and unsaturated features. Experimental results show that the ELM with the method of ReNLUs activation function, the accuracy and time of training and testing have been significantly improved and saved.