SPARSE CHOLESKY FACTORIZATION BY KULLBACK-LEIBLER MINIMIZATION

We propose to compute a sparse approximate inverse Cholesky factor L of a dense covariance matrix circle minus by minimizing the Kullback-Leibler divergence between the Gaussian distributions N (0, circle minus) and N (0, L-inverted perpendicular L-1), subject to a sparsity constraint. Surprisingly, this problem has a closed-form solution that can be computed efficiently, recovering the popular Vecchia approximation in spatial statistics. Based on recent results on the approximate sparsity of inverse Cholesky factors of circle minus obtained from pairwise evaluation of Green's functions of elliptic boundary-value problems at points {x(i)} 1 <= i <= N subset of R-d, we propose an elimination ordering and sparsity pattern that allows us to compute epsilon -approximate inverse Cholesky factors of such circle minus in computational complexity O (N log(N/epsilon)(d)) in space and O (N log(N/epsilon)(2d)) in time. To the best of our knowledge, this is the best asymptotic complexity for this class of problems. Furthermore, our method is embarrassingly parallel, automatically exploits low-dimensional structure in the data, and can perform Gaussian-process regression in linear (in N) space complexity. Motivated by its optimality properties, we propose applying our method to the joint covariance of training and prediction points in Gaussian-process regression, greatly improving stability and computational cost. Finally, we show how to apply our method to the important setting of Gaussian processes with additive noise, compromising neither accuracy nor computational complexity.