The complexity of first-order optimization methods from a metric perspective

被引:0
作者
Lewis, A. S. [1 ]
Tian, Tonghua [1 ]
机构
[1] Cornell Univ, ORIE, Ithaca, NY 14850 USA
基金
美国国家科学基金会;
关键词
Nonsmooth optimization and first-order algorithms; Slope; KL property; Complexity; Semi-algebraic; PROXIMAL POINT ALGORITHM; DESCENT METHODS; ERROR-BOUNDS; LOJASIEWICZ INEQUALITIES; GRADIENT FLOWS; CONVERGENCE; MINIMIZATION; SPACES;
D O I
10.1007/s10107-024-02091-2
中图分类号
TP31 [计算机软件];
学科分类号
081202 ; 0835 ;
摘要
A central tool for understanding first-order optimization algorithms is the Kurdyka-& Lstrok;ojasiewicz inequality. Standard approaches to such methods rely crucially on this inequality to leverage sufficient decrease conditions involving gradients or subgradients. However, the KL property fundamentally concerns not subgradients but rather "slope", a purely metric notion. By highlighting this view, and avoiding any use of subgradients, we present a simple and concise complexity analysis for first-order optimization algorithms on metric spaces. This subgradient-free perspective also frames a short and focused proof of the KL property for nonsmooth semi-algebraic functions.
引用
收藏
页码:49 / 78
页数:30
相关论文
共 50 条
[1]   Convergence of the iterates of descent methods for analytic cost functions [J].
Absil, PA ;
Mahony, R ;
Andrews, B .
SIAM JOURNAL ON OPTIMIZATION, 2005, 16 (02) :531-547
[2]  
Absil PA, 2008, OPTIMIZATION ALGORITHMS ON MATRIX MANIFOLDS, P1
[3]  
Ambrosio L, 2008, LECT MATH, P1
[4]   On the convergence of the proximal algorithm for nonsmooth functions involving analytic features [J].
Attouch, Hedy ;
Bolte, Jerome .
MATHEMATICAL PROGRAMMING, 2009, 116 (1-2) :5-16
[5]   Convergence of descent methods for semi-algebraic and tame problems: proximal algorithms, forward-backward splitting, and regularized Gauss-Seidel methods [J].
Attouch, Hedy ;
Bolte, Jerome ;
Svaiter, Benar Fux .
MATHEMATICAL PROGRAMMING, 2013, 137 (1-2) :91-129
[6]   Proximal Alternating Minimization and Projection Methods for Nonconvex Problems: An Approach Based on the Kurdyka-Lojasiewicz Inequality [J].
Attouch, Hedy ;
Bolte, Jerome ;
Redont, Patrick ;
Soubeyran, Antoine .
MATHEMATICS OF OPERATIONS RESEARCH, 2010, 35 (02) :438-457
[7]   A MOVING BALLS APPROXIMATION METHOD FOR A CLASS OF SMOOTH CONSTRAINED MINIMIZATION PROBLEMS [J].
Auslender, Alfred ;
Shefi, Ron ;
Teboulle, Marc .
SIAM JOURNAL ON OPTIMIZATION, 2010, 20 (06) :3232-3259
[8]   Characterizations of error bounds for lower semicontinuous functions on metric spaces [J].
Azé, D ;
Corvellec, JN .
ESAIM-CONTROL OPTIMISATION AND CALCULUS OF VARIATIONS, 2004, 10 (03) :409-425
[9]   Nonlinear Error Bounds via a Change of Function [J].
Aze, Dominique ;
Corvellec, Jean-Noel .
JOURNAL OF OPTIMIZATION THEORY AND APPLICATIONS, 2017, 172 (01) :9-32
[10]  
Bacak M., 2014, Convex analysis and optimization in Hadamard spaces