From Halpern's fixed-point iterations to Nesterov's accelerated interpretations for root-finding problems

被引:6
作者
Quoc, Tran-Dinh [1 ]
机构
[1] Univ North Carolina Chapel Hill, Dept Stat & Operat Res, 333 Hanes Hall,CB 3260,UNC, Chapel Hill, NC 27599 USA
基金
美国国家科学基金会;
关键词
Halpern's fixed-point iteration; Nesterov's accelerated method; Co-coercive equation; Maximally monotone inclusion; Extra-anchored gradient method; MONOTONE-OPERATORS; CONVERGENCE; ALGORITHMS; APPROXIMATION;
D O I
10.1007/s10589-023-00518-8
中图分类号
C93 [管理学]; O22 [运筹学];
学科分类号
070105 ; 12 ; 1201 ; 1202 ; 120202 ;
摘要
We derive an equivalent form of Halpern's fixed-point iteration scheme for solving a co-coercive equation (also called a root-finding problem), which can be viewed as a Nesterov's accelerated interpretation. We show that one method is equivalent to another via a simple transformation, leading to a straightforward convergence proof for Nesterov's accelerated scheme. Alternatively, we directly establish convergence rates of Nesterov's accelerated variant, and as a consequence, we obtain a new convergence rate of Halpern's fixed-point iteration. Next, we apply our results to different methods to solve monotone inclusions, where our convergence guarantees are applied. Since the gradient/forward scheme requires the co-coerciveness of the underlying operator, we derive new Nesterov's accelerated variants for both recent extra-anchored gradient and past-extra anchored gradient methods in the literature. These variants alleviate the co-coerciveness condition by only assuming the monotonicity and Lipschitz continuity of the underlying operator. Interestingly, our new Nesterov's accelerated interpretation of the past-extra anchored gradient method involves two past-iterate correction terms. This formulation is expected to guide us developing new Nesterov's accelerated methods for minimax problems and their continuous views without co-coericiveness. We test our theoretical results on two numerical examples, where the actual convergence rates match well the theoretical ones up to a constant factor.
引用
收藏
页码:181 / 218
页数:38
相关论文
共 56 条
[41]   A MODIFICATION OF THE ARROW-HURWICZ METHOD FOR SEARCH OF SADDLE POINTS [J].
POPOV, LD .
MATHEMATICAL NOTES, 1980, 28 (5-6) :845-848
[42]   MONOTONE OPERATORS AND PROXIMAL POINT ALGORITHM [J].
ROCKAFELLAR, RT .
SIAM JOURNAL ON CONTROL, 1976, 14 (05) :877-898
[43]  
Rockafellar RT, 1998, Variational Analysis
[44]  
RYU E. K., 2022, INT C MACHINE LEARNI, V162, P17420
[45]  
Ryu EK, 2016, APPL COMPUT MATH-BAK, V15, P3
[46]   A FIRST ORDER METHOD FOR SOLVING CONVEX BILEVEL OPTIMIZATION PROBLEMS [J].
Sabach, Shoham ;
Shtern, Shimrit .
SIAM JOURNAL ON OPTIMIZATION, 2017, 27 (02) :640-660
[47]   Understanding the acceleration phenomenon via high-resolution differential equations [J].
Shi, Bin ;
Du, Simon S. ;
Jordan, Michael, I ;
Su, Weijie J. .
MATHEMATICAL PROGRAMMING, 2022, 195 (1-2) :79-148
[48]  
Su WJ, 2014, ADV NEUR IN, V27
[49]  
Tran-Dinh Q, 2025, Arxiv, DOI arXiv:2301.03113
[50]  
Tran-Dinh Q, 2021, Arxiv, DOI arXiv:2110.08150