Inexact accelerated high-order proximal-point methods

被引:15
作者
Nesterov, Yurii [1 ]
机构
[1] Catholic Univ Louvain UCL, Ctr Operat Res & Econometr CORE, Louvain La Neuve, Belgium
基金
欧洲研究理事会;
关键词
Convex optimization; Tensor methods; Proximal-point operator; Lower complexity bounds; Optimal methods; 1ST-ORDER METHODS; OPTIMIZATION;
D O I
10.1007/s10107-021-01727-x
中图分类号
TP31 [计算机软件];
学科分类号
081202 ; 0835 ;
摘要
In this paper, we present a new framework of bi-level unconstrained minimization for development of accelerated methods in Convex Programming. These methods use approximations of the high-order proximal points, which are solutions of some auxiliary parametric optimization problems. For computing these points, we can use different methods, and, in particular, the lower-order schemes. This opens a possibility for the latter methods to overpass traditional limits of the Complexity Theory. As an example, we obtain a new second-order method with the convergence rate O (k(-4)),where k is the iteration counter. This rate is better than the maximal possible rate of convergence for this type of methods, as applied to functions with Lipschitz continuous Hessian. We also present newmethods with the exact auxiliary search procedure, which have the rate of convergence O (k(-(3p+1)/2)), where p >= 1 is the order of the proximal operator. The auxiliary problem at each iteration of these schemes is convex.
引用
收藏
页码:1 / 26
页数:26
相关论文
共 26 条
[1]  
[Anonymous], 2017, ARXIV171010329V1MATH
[2]  
Arjevani O.S., 2017, ARXIV170507260MATHOC
[3]  
Baes M., 2009, ESTIMATE SEQUENCE ME
[4]   A Descent Lemma Beyond Lipschitz Gradient Continuity: First-Order Methods Revisited and Applications [J].
Bauschke, Heinz H. ;
Bolte, Jerome ;
Teboulle, Marc .
MATHEMATICS OF OPERATIONS RESEARCH, 2017, 42 (02) :330-348
[5]   Worst-case evaluation complexity for unconstrained nonlinear optimization using high-order regularized models [J].
Birgin, E. G. ;
Gardenghi, J. L. ;
Martinez, J. M. ;
Santos, S. A. ;
Toint, Ph. L. .
MATHEMATICAL PROGRAMMING, 2017, 163 (1-2) :359-368
[6]  
Bubeck Sebastien, 2019, P MACHINE LEARNING R, V99
[7]   ACCELERATED METHODS FOR NONCONVEX OPTIMIZATION [J].
Carmon, Yair ;
Duchi, John C. ;
Hinder, Oliver ;
Sidford, Aaron .
SIAM JOURNAL ON OPTIMIZATION, 2018, 28 (02) :1751-1772
[8]   ON THE ORACLE COMPLEXITY OF FIRST-ORDER AND DERIVATIVE-FREE ALGORITHMS FOR SMOOTH NONCONVEX MINIMIZATION [J].
Cartis, Coralia ;
Gould, Nicholas I. M. ;
Toint, Philippe L. .
SIAM JOURNAL ON OPTIMIZATION, 2012, 22 (01) :66-86
[9]  
Gasnikov A., 2018, ARXIV180900382
[10]   NEW PROXIMAL POINT ALGORITHMS FOR CONVEX MINIMIZATION [J].
Gueler, Osman .
SIAM JOURNAL ON OPTIMIZATION, 1992, 2 (04) :649-664