Probabilistic time series forecasting with deep non-linear state space models

Probabilistic time series forecasting aims at estimating future probabilistic distributions based on given time series observations. It is a widespread challenge in various tasks, such as risk management and decision making. To investigate temporal patterns in time series data and predict subsequent probabilities, the state space model (SSM) provides a general framework. Variants of SSM achieve considerable success in many fields, such as engineering and statistics. However, since underlying processes in real-world scenarios are usually unknown and complicated, actual time series observations are always irregular and noisy. Therefore, it is very difficult to determinate an SSM for classical statistical approaches. In this paper, a general time series forecasting framework, called Deep Non-linear State Space Model (DNLSSM), is proposed to predict the probabilistic distribution based on estimated underlying unknown processes from historical time series data. We fuse deep neural networks and statistical methods to iteratively estimate states and network parameters and thus exploit intricate temporal patterns of time series data. In particular, the unscented Kalman filter (UKF) is adopted to calculate marginal likelihoods and update distributions recursively for non-linear functions. After that, a non-linear Joseph form covariance update is developed to ensure that calculated covariance matrices in UKF updates are symmetric and positive definitive. Therefore, the authors enhance the tolerance of UKF to round-off errors and manage to combine UKF and deep neural networks. In this manner, the DNLSSM effectively models non-linear correlations between observed time series data and underlying dynamic processes. Experiments in both synthetic and real-world datasets demonstrate that the DNLSSM consistently improves the accuracy of probability forecasts compared to the baseline methods.