Growing deep echo state network with supervised learning for time series prediction

Multilayer echo state networks (ESNs) are powerful on learning hierarchical temporal representation. However, how to determine the depth of multilayer ESNs is still an open issue. In this paper, we propose a novel approach to automatically determine the depth of a multilayer ESN, named growing deep ESN (GD-ESN). First, an incremental hierarchical structure is proposed, where the recurrent layers and the pre-trained feedforward layers are alternately added to the network one by one. Then, a control scheme is designed for the growth of the network based on the newly defined averaged mutual information and the full rank criterion. Finally, the proposed GD-ESN is evaluated on both benchmark datasets and real-world applications. The experimental results show the effectiveness of the proposed method. (C) 2022 Elsevier B.V. All rights reserved.