Generating set search using simplex gradients for bound-constrained black-box optimization

The optimization problems arising in modern engineering practice are increasingly simulation-based, characterized by extreme types of nonsmoothness, the inaccessibility of derivatives, and high computational expense. While generating set searches (GSS) generally offer a satisfying level of robustness and converge to stationary points, the convergence rates may be slow. In order to accelerate the solution process without sacrificing robustness, we introduce (simplex) gradient-informed generating set search (GIGS) methods for solving bound-constrained minimization problems. These algorithms use simplex gradients, acquired over several iterations, as guidance for adapting the search stencil to the local topography of the objective function. GIGS is shown to inherit first-order convergence properties of GSS and to possess a natural tendency for avoiding saddle points. Numerical experiments are performed on an academic set of smooth, nonsmooth and noisy test problems, as well as a realistic engineering case study. The results demonstrate that including simplex gradient information enables computational cost savings over non-adaptive GSS methods.