The complexity of first-order optimization methods from a metric perspective

被引:0
作者
Lewis, A. S. [1 ]
Tian, Tonghua [1 ]
机构
[1] Cornell Univ, ORIE, Ithaca, NY 14850 USA
基金
美国国家科学基金会;
关键词
Nonsmooth optimization and first-order algorithms; Slope; KL property; Complexity; Semi-algebraic; PROXIMAL POINT ALGORITHM; DESCENT METHODS; ERROR-BOUNDS; LOJASIEWICZ INEQUALITIES; GRADIENT FLOWS; CONVERGENCE; MINIMIZATION; SPACES;
D O I
10.1007/s10107-024-02091-2
中图分类号
TP31 [计算机软件];
学科分类号
081202 ; 0835 ;
摘要
A central tool for understanding first-order optimization algorithms is the Kurdyka-& Lstrok;ojasiewicz inequality. Standard approaches to such methods rely crucially on this inequality to leverage sufficient decrease conditions involving gradients or subgradients. However, the KL property fundamentally concerns not subgradients but rather "slope", a purely metric notion. By highlighting this view, and avoiding any use of subgradients, we present a simple and concise complexity analysis for first-order optimization algorithms on metric spaces. This subgradient-free perspective also frames a short and focused proof of the KL property for nonsmooth semi-algebraic functions.
引用
收藏
页数:30
相关论文
共 50 条
  • [21] Bridson Haeiger A., 1999, GRUND MATH WISS, V319, DOI [DOI 10.1007/978-3-662-12494-9, 10.1007/978-3-662-12494-9]
  • [22] PROXIMAL GRADIENT METHOD FOR NONSMOOTH OPTIMIZATION OVER THE STIEFEL MANIFOLD
    Chen, Shixiang
    Ma, Shiqian
    So, Anthony Man-Cho
    Zhang, Tong
    [J]. SIAM JOURNAL ON OPTIMIZATION, 2020, 30 (01) : 210 - 239
  • [23] Coste M., 1999, RAAG NOTES
  • [24] De Giorgi E., 1980, ATTI ACCAD NAZ SFMN, V68, P180
  • [25] MAXIMUM LIKELIHOOD FROM INCOMPLETE DATA VIA EM ALGORITHM
    DEMPSTER, AP
    LAIRD, NM
    RUBIN, DB
    [J]. JOURNAL OF THE ROYAL STATISTICAL SOCIETY SERIES B-METHODOLOGICAL, 1977, 39 (01): : 1 - 38
  • [26] Nonsmooth optimization using Taylor-like models: error bounds, convergence, and termination criteria
    Drusvyatskiy, D.
    Ioffe, A. D.
    Lewis, A. S.
    [J]. MATHEMATICAL PROGRAMMING, 2021, 185 (1-2) : 357 - 383
  • [27] CURVES OF DESCENT
    Drusvyatskiy, D.
    Ioffe, A. D.
    Lewis, A. S.
    [J]. SIAM JOURNAL ON CONTROL AND OPTIMIZATION, 2015, 53 (01) : 114 - 138
  • [28] VARIATIONAL PRINCIPLE
    EKELAND, I
    [J]. JOURNAL OF MATHEMATICAL ANALYSIS AND APPLICATIONS, 1974, 47 (02) : 324 - 353
  • [29] GULER O, 1991, SIAM J CONTROL OPTIM, V29, P403, DOI 10.1137/0329022
  • [30] KURDYKA-LOJASIEWICZ-SIMON INEQUALITY FOR GRADIENT FLOWS IN METRIC SPACES
    Hauer, Daniel
    Mazon, Jose M.
    [J]. TRANSACTIONS OF THE AMERICAN MATHEMATICAL SOCIETY, 2019, 372 (07) : 4917 - 4976