Online Recurrent Extreme Learning Machine and its Application to Time-series Prediction

Online sequential extreme learning machine (OSELM) is an online learning algorithm training single-hidden layer feedforward neural networks (SLFNs), which can learn data one-by-one or chunk-by-chunk with fixed or varying data size. Due to its characteristics of online sequential learning, OS-ELM is popularly used to solve time-series prediction problem, such as stock forecast, weather forecast, passenger count forecast, etc. OS-ELM, however, has two fatal drawbacks: Its input weights cannot be adjusted and it cannot be applied to learn recurrent neural network (RNN). Therefore we propose a modified version of OS-ELM, called online recurrent extreme learning machine (OR-ELM), which is able to adjust input weights and can be applied to learn RNN, by applying ELM-auto-encoder and a normalization method called layer normalization (LN). Proposed method is used to solve a time-series prediction problem on NewYork City passenger count dataset, and the results show that R-ELM outperforms OS-ELM and other online-sequential learning algorithms such as hierarchical temporal memory (HTM) and online long short-term memory (online LSTM).