Clustered and deep echo state networks for signal noise reduction

Echo State Networks (ESNs) are Recurrent Neural Networks with fixed input and internal (hidden) weights, and adaptable output weights. The hidden part of an ESN can be considered as a discrete-time dynamical system, called reservoir. In classical ESNs, the internal connections are obtained from an Erdos-Renyi graph. A recent study proposed ESNs with clustered adjacency matrices (CESNs), where the clusters are either Erdos-Renyi graphs or Barabasi-Albert-like graphs. In this work, we investigate the effectiveness of CESNs and apply them for signal denoising. In addition, we introduce and study deep CESNs with multiple clustered layers. We found that CESNs and deep CESNs can compete with deep ESNs for all tasks that we considered.