Exact Bayesian inference via data augmentation

Data augmentation is a common tool in Bayesian statistics, especially in the application of MCMC. Data augmentation is used where direct computation of the posterior density, pi(theta|x), of the parameters theta, given the observed data x, is not possible. We show that for a range of problems, it is possible to augment the data by y, such that, pi(theta|x,y) is known, and pi(y|x) can easily be computed. In particular, pi(y|x) is obtained by collapsing pi(y,theta|x) through integrating out theta. This allows the exact computation of pi(theta|x) as a mixture distribution without recourse to approximating methods such as MCMC. Useful byproducts of the exact posterior distribution are the marginal likelihood of the model and the exact predictive distribution.