Exact Gaussian Process Regression with Distributed Computations

Gaussian Processes (GPs) are powerful non-parametric Bayesian models for function estimation, but suffer from high complexity in terms of both computation and storage. To address such issues, approximation methods have flourished in the literature, including model approximations and approximate inference. However, these methods often sacrifice accuracy for scalability. In this work, we present the design and evaluation of a distributed method for exact GP inference, that achieves true model parallelism using simple, high-level distributed computing frameworks. Our experiments show that exact inference at scale is not only feasible, but it also brings substantial benefits in terms of low error rates and accurate quantification of uncertainty.