Practical initialization of the Nelder-Mead method for computationally expensive optimization problems

被引:18
作者
Takenaga, Shintaro [1 ,2 ]
Ozaki, Yoshihiko [2 ,3 ]
Onishi, Masaki [2 ]
机构
[1] Univ Tsukuba, Ibaraki, Japan
[2] AIST, AI Res Ctr, Tokyo, Japan
[3] GREE Inc, Tokyo, Japan
关键词
Nelder-Mead method; Initialization; Computationally expensive optimization; Black-box optimization;
D O I
10.1007/s11590-022-01953-y
中图分类号
C93 [管理学]; O22 [运筹学];
学科分类号
070105 ; 12 ; 1201 ; 1202 ; 120202 ;
摘要
Black-box optimization (BBO) algorithms are widely employed by practitioners to address computationally expensive real-world problems such as automatic tuning of machine learning models and evacuation route planning. The Nelder-Mead (NM) method is a well-known local search heuristic for BBO that has been applied to solve many real-world problems from way back because of its promising performance. However, this method has a strong dependence on initialization due to its local search tendency. Nevertheless, a discussion on the proper initialization of the NM method is limited to the recent study by Wessing (Optim Lett 13(4):847-856, 2019), which is solely based on an analysis using the simple sphere function. In this study, we take a further step to improve Wessing's result by massively investigating how the initialization affects the search performance in views of the initial simplex size and shape and a constraint handling method that is employed on 24 BBO benchmarking problems. Based on the numerical results, we present the empirical best practice for the initialization of the NM method for cases involving a limited evaluation budget.
引用
收藏
页码:283 / 297
页数:15
相关论文
共 26 条
[1]  
Audet C., 2017, Springer Series in Operations Research and Financial Engineering, DOI [10.1007/978-3-319-68913-5, DOI 10.1007/978-3-319-68913-5]
[2]  
Cohen G, 2005, FLAIRS C, P431
[3]  
Conn AR, 2009, MOS-SIAM SER OPTIMIZ, V8, P1
[4]  
Digabel S.L., 2015, ARXIV
[5]  
EFRON B, 1987, J AM STAT ASSOC, V82, P171, DOI 10.2307/2289144
[6]  
Efron B, 1982, The Jackknife, the Bootstrap and Other Resembling Plans
[7]  
Fan E., 2002, GLOBAL OPTIMIZATION
[8]  
Finck S., 2009, Real-parameter black-box optimization benchmarking 2010: Presentation of the noiseless functions
[9]   Implementing the Nelder-Mead simplex algorithm with adaptive parameters [J].
Gao, Fuchang ;
Han, Lixing .
COMPUTATIONAL OPTIMIZATION AND APPLICATIONS, 2012, 51 (01) :259-277
[10]   Constrained globalized Nelder-Mead method for simultaneous structural and manufacturing optimization of a composite bracket [J].
Ghiasi, Hossein ;
Pasini, Damiano ;
Lessard, Larry .
JOURNAL OF COMPOSITE MATERIALS, 2008, 42 (07) :717-736