Big data regression with parallel enhanced and convex incremental extreme learning machines

This work considers scalable incremental extreme learning machine (I-ELM) algorithms, which could be suitable for big data regression. During the training of I-ELMs, the hidden neurons are presented one by one, and the weights are based solely on simple direct summations, which can be most efficiently mapped on parallel environments. Existing incremental versions of ELMs are the I-ELM, enhanced incremental ELM (EI-ELM), and convex incremental ELM (CI-ELM). We study the enhanced and convex incremental ELM (ECI-ELM) algorithm, which is a combination of the last 2 versions. The main findings are that ECI-ELM is fast, accurate, and fully scalable when it operates in a parallel system of distributed memory workstations. Experimental simulations on several benchmark data sets demonstrate that the ECI-ELM is the most accurate among the existing I-ELM, EI-ELM, and CI-ELM algorithms. We also analyze the convergence as a function of the hidden neurons and demonstrate that ECI-ELM has the lowest error rate curve and converges much faster than the other algorithms in all of the data sets. The parallel simulations also reveal that the data parallel training of the ECI-ELM can guarantee simplicity and straightforward mappings and can deliver speedups and scale-ups very close to linear.