Context Dependent Input Weight Selection for Regression Extreme Learning Machines

Extreme learning machine (ELM) is a popular machine learning algorithm due to its fast non-iterative training and good generalization [2]. However, it randomly assigns input weights and biases from a uniform distribution regardless of the characteristics of the training data. Exploiting this data would produce a more specialized model, instead of adopting a "one size fits all" approach. This could result in better generalization while preserving ELM's fast training. Hence, we developed a context dependent input weight selection for regression ELM (CDR-ELM) which is a non-iterative training algorithm for supervised regression. First, k-means clusters input data into P clusters based on the number of hidden layer neurons. Then, cluster head differences are assigned to input weights as described in (1), and biases are computed from cluster sizes as b(i) = N-j/N-k. Finally, ELM is trained using least squares.