From Halpern's fixed-point iterations to Nesterov's accelerated interpretations for root-finding problems

被引:6
作者
Quoc, Tran-Dinh [1 ]
机构
[1] Univ North Carolina Chapel Hill, Dept Stat & Operat Res, 333 Hanes Hall,CB 3260,UNC, Chapel Hill, NC 27599 USA
基金
美国国家科学基金会;
关键词
Halpern's fixed-point iteration; Nesterov's accelerated method; Co-coercive equation; Maximally monotone inclusion; Extra-anchored gradient method; MONOTONE-OPERATORS; CONVERGENCE; ALGORITHMS; APPROXIMATION;
D O I
10.1007/s10589-023-00518-8
中图分类号
C93 [管理学]; O22 [运筹学];
学科分类号
070105 ; 12 ; 1201 ; 1202 ; 120202 ;
摘要
We derive an equivalent form of Halpern's fixed-point iteration scheme for solving a co-coercive equation (also called a root-finding problem), which can be viewed as a Nesterov's accelerated interpretation. We show that one method is equivalent to another via a simple transformation, leading to a straightforward convergence proof for Nesterov's accelerated scheme. Alternatively, we directly establish convergence rates of Nesterov's accelerated variant, and as a consequence, we obtain a new convergence rate of Halpern's fixed-point iteration. Next, we apply our results to different methods to solve monotone inclusions, where our convergence guarantees are applied. Since the gradient/forward scheme requires the co-coerciveness of the underlying operator, we derive new Nesterov's accelerated variants for both recent extra-anchored gradient and past-extra anchored gradient methods in the literature. These variants alleviate the co-coerciveness condition by only assuming the monotonicity and Lipschitz continuity of the underlying operator. Interestingly, our new Nesterov's accelerated interpretation of the past-extra anchored gradient method involves two past-iterate correction terms. This formulation is expected to guide us developing new Nesterov's accelerated methods for minimax problems and their continuous views without co-coericiveness. We test our theoretical results on two numerical examples, where the actual convergence rates match well the theoretical ones up to a constant factor.
引用
收藏
页码:181 / 218
页数:38
相关论文
共 56 条
[1]  
[Anonymous], 1976, Matecon
[2]   FROM THE RAVINE METHOD TO THE NESTEROV METHOD AND VICE VERSA: A DYNAMICAL SYSTEM PERSPECTIVE [J].
Attouch, Hedy ;
Fadili, Jalal .
SIAM JOURNAL ON OPTIMIZATION, 2022, 32 (03) :2074-2101
[3]   Convergence of a relaxed inertial proximal algorithm for maximally monotone operators [J].
Attouch, Hedy ;
Cabot, Alexandre .
MATHEMATICAL PROGRAMMING, 2020, 184 (1-2) :243-287
[4]   Convergence of inertial dynamics and proximal algorithms governed by maximally monotone operators [J].
Attouch, Hedy ;
Peypouquet, Juan .
MATHEMATICAL PROGRAMMING, 2019, 174 (1-2) :391-432
[5]   THE RATE OF CONVERGENCE OF NESTEROV'S ACCELERATED FORWARD-BACKWARD METHOD IS ACTUALLY FASTER THAN 1/k2 [J].
Attouch, Hedy ;
Peypouquet, Juan .
SIAM JOURNAL ON OPTIMIZATION, 2016, 26 (03) :1824-1834
[6]   Generalized monotone operators and their averaged resolvents [J].
Bauschke, Heinz H. ;
Moursi, Walaa M. ;
Wang, Xianfu .
MATHEMATICAL PROGRAMMING, 2021, 189 (1-2) :55-74
[7]  
Bauschke HH, 2011, CMS BOOKS MATH, P1, DOI 10.1007/978-1-4419-9467-7
[8]   The approximation of fixed points of compositions of nonexpansive mappings in Hilbert space [J].
Bauschke, HH .
JOURNAL OF MATHEMATICAL ANALYSIS AND APPLICATIONS, 1996, 202 (01) :150-159
[9]   A Fast Iterative Shrinkage-Thresholding Algorithm for Linear Inverse Problems [J].
Beck, Amir ;
Teboulle, Marc .
SIAM JOURNAL ON IMAGING SCIENCES, 2009, 2 (01) :183-202
[10]  
Bot R.I., 2022, arXiv