THE RATE OF CONVERGENCE OF NESTEROV'S ACCELERATED FORWARD-BACKWARD METHOD IS ACTUALLY FASTER THAN 1/k2

被引:215
作者
Attouch, Hedy [1 ]
Peypouquet, Juan [2 ]
机构
[1] Univ Montpellier 2, Inst Math & Modelisat Montpellier, CNRS, Pl Eugene Bataillon, F-34095 Montpellier 5, France
[2] Univ Tecn Federico Santa Maria, Dept Matemat, Ave Espana 1680, Valparaiso, Chile
关键词
convex optimization; fast convergent methods; Nesterov method; INERTIAL PROXIMAL METHOD; LINEAR INVERSE PROBLEMS; MONOTONE-OPERATORS; THRESHOLDING ALGORITHM; HILBERT-SPACE; SUM;
D O I
10.1137/15M1046095
中图分类号
O29 [应用数学];
学科分类号
070104 ;
摘要
The forward-backward algorithm is a powerful tool for solving optimization problems with an additively separable and smooth plus nonsmooth structure. In the convex setting, a simple but ingenious acceleration scheme developed by Nesterov improves the theoretical rate of convergence for the function values from the standard O(k(-1)) down to O(k(-2)). In this short paper, we prove that the rate of convergence of a slight variant of Nesterov's accelerated forward-backward method, which produces convergent sequences, is actually o(k(-2)), rather than O(k(-2)). Our arguments rely on the connection between this algorithm and a second-order differential inclusion with vanishing damping.
引用
收藏
页码:1824 / 1834
页数:11
相关论文
共 26 条
[11]   CONVEX PROGRAMMING IN HILBERT SPACE [J].
GOLDSTEIN, AA .
BULLETIN OF THE AMERICAN MATHEMATICAL SOCIETY, 1964, 70 (05) :709-&
[12]   Optimized first-order methods for smooth convex minimization [J].
Kim, Donghwan ;
Fessler, Jeffrey A. .
MATHEMATICAL PROGRAMMING, 2016, 159 (1-2) :81-107
[13]  
Levitin ES., 1966, USSR COMP MATH MATH, V6, P1, DOI [DOI 10.1016/0041-5553(66)90114-5, 10.1016/0041-5553(66)90114-5]
[14]   SPLITTING ALGORITHMS FOR THE SUM OF 2 NON-LINEAR OPERATORS [J].
LIONS, PL ;
MERCIER, B .
SIAM JOURNAL ON NUMERICAL ANALYSIS, 1979, 16 (06) :964-979
[15]  
May R., ARXIV150905598
[16]   Convergence of a splitting inertial proximal method for monotone operators [J].
Moudafi, A ;
Oliny, M .
JOURNAL OF COMPUTATIONAL AND APPLIED MATHEMATICS, 2003, 155 (02) :447-454
[17]   Smooth minimization of non-smooth functions [J].
Nesterov, Y .
MATHEMATICAL PROGRAMMING, 2005, 103 (01) :127-152
[18]  
Nesterov Y, 2013, Lectures on convex optimization
[19]  
Nesterov Y., 2007, CORE DISCUSSION PAPE
[20]  
Nesterov Y., 1983, DOKL AKAD NAUK SSSR, V27, P372