The complexity of first-order optimization methods from a metric perspective

被引:0
作者
Lewis, A. S. [1 ]
Tian, Tonghua [1 ]
机构
[1] Cornell Univ, ORIE, Ithaca, NY 14850 USA
基金
美国国家科学基金会;
关键词
Nonsmooth optimization and first-order algorithms; Slope; KL property; Complexity; Semi-algebraic; PROXIMAL POINT ALGORITHM; DESCENT METHODS; ERROR-BOUNDS; LOJASIEWICZ INEQUALITIES; GRADIENT FLOWS; CONVERGENCE; MINIMIZATION; SPACES;
D O I
10.1007/s10107-024-02091-2
中图分类号
TP31 [计算机软件];
学科分类号
081202 ; 0835 ;
摘要
A central tool for understanding first-order optimization algorithms is the Kurdyka-& Lstrok;ojasiewicz inequality. Standard approaches to such methods rely crucially on this inequality to leverage sufficient decrease conditions involving gradients or subgradients. However, the KL property fundamentally concerns not subgradients but rather "slope", a purely metric notion. By highlighting this view, and avoiding any use of subgradients, we present a simple and concise complexity analysis for first-order optimization algorithms on metric spaces. This subgradient-free perspective also frames a short and focused proof of the KL property for nonsmooth semi-algebraic functions.
引用
收藏
页码:49 / 78
页数:30
相关论文
共 50 条
[21]  
Bridson Haeiger A., 1999, GRUND MATH WISS, V319, DOI DOI 10.1007/978-3-662-12494-9
[22]   PROXIMAL GRADIENT METHOD FOR NONSMOOTH OPTIMIZATION OVER THE STIEFEL MANIFOLD [J].
Chen, Shixiang ;
Ma, Shiqian ;
So, Anthony Man-Cho ;
Zhang, Tong .
SIAM JOURNAL ON OPTIMIZATION, 2020, 30 (01) :210-239
[23]  
Coste M., 1999, RAAG NOTES
[24]  
De Giorgi E., 1980, ATTI ACCAD NAZ LINCE, V68, P180
[25]   MAXIMUM LIKELIHOOD FROM INCOMPLETE DATA VIA EM ALGORITHM [J].
DEMPSTER, AP ;
LAIRD, NM ;
RUBIN, DB .
JOURNAL OF THE ROYAL STATISTICAL SOCIETY SERIES B-METHODOLOGICAL, 1977, 39 (01) :1-38
[26]   Nonsmooth optimization using Taylor-like models: error bounds, convergence, and termination criteria [J].
Drusvyatskiy, D. ;
Ioffe, A. D. ;
Lewis, A. S. .
MATHEMATICAL PROGRAMMING, 2021, 185 (1-2) :357-383
[27]   CURVES OF DESCENT [J].
Drusvyatskiy, D. ;
Ioffe, A. D. ;
Lewis, A. S. .
SIAM JOURNAL ON CONTROL AND OPTIMIZATION, 2015, 53 (01) :114-138
[28]   VARIATIONAL PRINCIPLE [J].
EKELAND, I .
JOURNAL OF MATHEMATICAL ANALYSIS AND APPLICATIONS, 1974, 47 (02) :324-353
[29]  
GULER O, 1991, SIAM J CONTROL OPTIM, V29, P403, DOI 10.1137/0329022
[30]   KURDYKA-LOJASIEWICZ-SIMON INEQUALITY FOR GRADIENT FLOWS IN METRIC SPACES [J].
Hauer, Daniel ;
Mazon, Jose M. .
TRANSACTIONS OF THE AMERICAN MATHEMATICAL SOCIETY, 2019, 372 (07) :4917-4976