An introduction to Majorization-Minimization algorithms for machine learning and statistical estimation

MM (majorization-minimization) algorithms are an increasingly popular tool for solving optimization problems in machine learning and statistical estimation. This article introduces the MM algorithm framework in general and via three commonly considered example applications: Gaussian mixture regressions, multinomial logistic regressions, and support vector machines. Specific algorithms for these three examples are derived and Mathematical Programming Series A numerical demonstrations are presented. Theoretical and practical aspects of MM algorithm design are discussed. (C) 2017 John Wiley & Sons, Ltd