Incremental Extreme Learning Machine based on Cascade Neural Networks

This paper extends extreme learning machine (ELM) for multi-layer cascade neural networks. We reformulate the cascade neural networks as a linear-in-the-parameters model, and propose a novel constructive training algorithm motivated by the efficient incremental ELM. The orthogonal least squares (OLS) is introduced to derive a new criterion for evaluating candidate hidden units, which avoids the computation of Moore-Penrose generalized inverse in the training process. Moreover, the calculation of output weights can be greatly simplified. Besides its efficiency, we show that the proposed evaluation function can effectively identify optimal candidate unit which leads to maximum error (sum of squared errors, SSE) reduction of the network. As a result, the proposed algorithm tends to yield smaller network with better generalization performance compared to traditional ELM. The effectiveness of the proposed algorithm on classification and regression problems is demonstrated by experimental results on several real-world datasets.