A Derivative-Free Optimization Algorithm Combining Line-Search and Trust-Region Techniques

被引：0
作者：
Pengcheng Xie
Ya-xiang Yuan
机构：
[1] University of Chinese Academy of Sciences,State Key Laboratory of Scientific/Engineering Computing, Institute of Computational Mathematics and Scientific/Engineering Computing, Academy of Mathematics and Systems Science, Chinese Academy of Sciences
来源：
Chinese Annals of Mathematics, Series B | 2023年 / 44卷
关键词：
Nonlinear optimization; Derivative-Free; Quadratic model; Line-Search; Trust-Region; 90C56; 90C30; 65K05; 90C90;
D O I：
暂无
中图分类号：
学科分类号：
摘要：
The speeding-up and slowing-down (SUSD) direction is a novel direction, which is proved to converge to the gradient descent direction under some conditions. The authors propose the derivative-free optimization algorithm SUSD-TR, which combines the SUSD direction based on the covariance matrix of interpolation points and the solution of the trust-region subproblem of the interpolation model function at the current iteration step. They analyze the optimization dynamics and convergence of the algorithm SUSD-TR. Details of the trial step and structure step are given. Numerical results show their algorithm’s efficiency, and the comparison indicates that SUSD-TR greatly improves the method’s performance based on the method that only goes along the SUSD direction. Their algorithm is competitive with state-of-the-art mathematical derivative-free optimization algorithms.
引用
收藏
页码：719 / 734
页数：15
相关论文
共 33 条
[1]  
Al-Abri S(2020)A derivative-free optimization method with application to functions with exploding and vanishing gradients IEEE Contr. Syst. Lett. 5 587-592
[2]  
Lin T X(2008)Response surface methodology (RSM) as a tool for optimization in analytical chemistry Talanta 76 965-977
[3]  
Tao M(2008)Geometry of sample sets in derivative free optimization, part II: Polynomial regression and underdetermined interpolation IMA J. Numer. Anal. 28 721-748
[4]  
Zhang F(1972)Quasi-Newton methods for unconstrained optimization IMA J. of Appl. Math. 9 91-108
[5]  
Bezerra M(1983)Computing forward-difference intervals for numerical optimization SIAM J. Sci. Statist. Comput. 4 310-321
[6]  
Santelli R(2003)Optimization by direct search: New perspectives on some classical and modern methods SIAM Rev. 45 385-482
[7]  
Oliveira E(2019)Derivative-free optimization methods Acta Numer. 28 287-404
[8]  
Conn A R(2009)Benchmarking derivative-free optimization algorithms SIAM J. Optim. 20 172-191
[9]  
Scheinberg K(1965)A simplex method for function minimization Comput. J. 7 308-313
[10]  
Vicente L N(2002)UOBYQA: Unconstrained optimization by quadratic approximation Math. Program. 92 555-582
1234