Time series prediction by feedforward neural networks - is it difficult?

The difficulties that a neural network faces when trying to learn from a quasiperiodic time series are studied analytically using a teacher-student scenario where the random input is divided into two macroscopic regions with different variances, 1 and 1/gamma(2) (gamma much greater than 1). The generalization error is found to decrease as is an element of(g) proportional to exp(-alpha/gamma(2)), where alpha is the number of examples per input dimension. In contradiction to this very slow vanishing generalization error, the next output prediction is found to be almost free of mistakes. This picture is consistent with learning quasi-periodic time series produced by feedforward neural networks, which is dominated by enhanced components of the Fourier spectrum of the input. Simulation results are in good agreement with the analytical results.