On unique representations of certain dynamical systems produced by continuous-time recurrent neural networks

This article extends previous mathematical studies on elucidating the redundancy for describing functions by feedforward neural networks (FNNs) to the elucidation of redundancy for describing dynamical systems (DSs) by continuous-time recurrent neural networks (RNNs). In order to approximate a DS on R" using an RNN with n visible units, an n-dimensional affine neural dynamical system (A-NDS) can be used as the DS actually produced by the above RNN under an affine map from its visible state-space R" to its hidden state-space. Therefore, we consider the problem of clarifying the redundancy for describing A-NDSs by RNNs and affine maps. We clarify to what extent a pair of an RNN and an affine map is uniquely determined by its corresponding A-NDS and also give a nonredundant sufficient search set for the DS approximation problem based on A-NDS.