Gaussian Variational Approximations for High-dimensional State Space Models

We consider a Gaussian variational approximation of the posterior density in high-dimensional state space models. The number of parameters in the covariance matrix of the variational approximation grows as the square of the number of model parameters, so it is necessary to find simple yet effective parametrisations of the covariance structure when the number of model parame-ters is large. We approximate the joint posterior density of the state vectors by a dynamic factor model, having Markovian time dependence and a factor covariance structure for the states. This gives a reduced description of the dependence struc-ture for the states, as well as a temporal conditional independence structure sim-ilar to that in the true posterior. We illustrate the methodology on two examples. The first is a spatio-temporal model for the spread of the Eurasian collared-dove across North America. Our approach compares favorably to a recently proposed ensemble Kalman filter method for approximate inference in high-dimensional hi-erarchical spatio-temporal models. Our second example is a Wishart-based multi-variate stochastic volatility model for financial returns, which is outside the class of models the ensemble Kalman filter method can handle.