Robust recurrent neural networks for time series forecasting

Recurrent neural networks (RNNs) are widely utilized in time series forecasting tasks. In practical appli-cations, there are noises in real-life time series data. A model's generalization capacity will be diminished by model uncertainty with regard to input noises. However, the robustness of RNNs with respect to input noises has not been well studied yet. The localized stochastic sensitivity (LSS), which measures output disturbances with respect to input perturbations of learning models, has been successfully applied to improve the robustness of different neural networks on tabular and image data. But its effectiveness on time series data has not been explored. Therefore, we extend the idea of LSS and apply it to the robust RNNs training for time series forecasting problems. With the minimization of LSS, output sensitivities of RNNs with respect to small perturbations are reduced. So, the proposed robust RNNs will not be affected by slight input noises. We have used the LSTM as an example for theoretical analysis and analyzed the effectiveness of LSS on several RNN variants including the vanilla RNN, the gated recurrent unit (GRU), the long-short-term memory (LSTM), and the bi-directional long short-term memory (Bi-LSTM) in empir-ical studies. Experimental results confirm the efficiency of applying LSS to enhance the robustness of RNNs for time series data. For example, the robust RNNs outperform their counterpart by 53.26% and 49.45% on average in terms of Root Mean Squared Error and R-Square on five datasets.(c) 2023 Elsevier B.V. All rights reserved.