A Computationally Efficient Limited Memory CMA-ES for Large Scale Optimizat

We propose a computationally efficient limited memory Covariance Matrix Adaptation Evolution Strategy for large scale optimization, which we call the LM-CMA-ES. The LM-CMA-ES is a stochastic, derivative-free algorithm for numerical optimization of non-linear, non-convex optimization problems in continuous domain. Inspired by the limited memory BFGS method of Liu and Nocedal (1989), the LM-CMA-ES samples candidate solutions according to a covariance matrix reproduced from m direction vectors selected during the optimization process. The decomposition of the covariance matrix into Cholesky factors allows to reduce the time and memory complexity of the sampling to O (mn), where n is the number of decision variables. When n is large (e.g., n > 1000), even relatively small values of m (e.g., m = 20; 30) are sufficient to efficiently solve fully nonseparable problems and to reduce the overall run-time.