Accelerating the convergence of the EM algorithm using the vector ε algorithm

The EM algorithm of Dempster, Laird and Rubin [1977. Maximum likelihood from incomplete data via the EM algorithm. J. Roy. Statist. Soc. Ser. B 39, 1-22] is a very general and popular iterative computational algorithm that is used to find maximum likelihood estimates from incomplete data and is widely used to perform statistical analysis with missing data, because of its stability, flexibility and simplicity. However, a common criticism is that the convergence of the EM algorithm is slow. Various algorithms to accelerate the convergence of the EM algorithm have been proposed. In this paper, we propose the "epsilon-accelerated EM algorithm" that speeds up the convergence of the EM sequence via the vector epsilon algorithm of Wynn [1962. Acceleration techniques for iterated vector and matrix problems. Math. Comp. 16, 304-322]. We also demonstrate its theoretical properties. The epsilon-accelerated EM algorithm has been successfully extended to the EM algorithm without affecting its stability, flexibility and simplicity. Numerical experiments illustrate the potential of the epsilon-accelerated EM algorithm. (c) 2006 Elsevier B.V. All rights reserved.