Effective matrix adaptation strategy for noisy derivative-free optimization

In this paper, we introduce a new effective matrix adaptation evolution strategy (MADFO) for noisy derivative-free optimization problems. Like every MAES solver, MADFO consists of three phases: mutation, selection and recombination. MADFO improves the mutation phase by generating good step sizes, neither too small not too large, that increase the probability of selecting mutation points with small inexact function values in the selection phase. In the recombination phase, a recombination point with lowest inexact function value found among all evaluated points so far may be found by a new randomized non-monotone line search method and accepted as the best point. If no best point is found, a heuristic point may be accepted as the best point. We compare MADFO with state-of-the-art DFO solvers on noisy test problems obtained by adding various kinds and levels of noise to all unconstrained CUTEst test problems with dimensions n <= 20\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$n\le 20$$\end{document}, and find that MADFO has the highest number of solved problems