Generalized Nesterov’s accelerated proximal gradient algorithms with convergence rate of order o(1/k2)

被引:0
作者
Huynh Van Ngai
Ta Anh Son
机构
[1] University of Quy Nhon,Department of Mathematics and Statistics
[2] Ha Noi University of Science and Technology,School of Applied Mathematics and Informatics
来源
Computational Optimization and Applications | 2022年 / 83卷
关键词
Convex optimization; Forward–backward method; Nesterov accelerated gradient method; Proximal mapping; Subdifferential; 49J52; 90C26; 90C30; 49M37; 65K05; 90C25;
D O I
暂无
中图分类号
学科分类号
摘要
The accelerated gradient method initiated by Nesterov is now recognized to be one of the most powerful tools for solving smooth convex optimization problems. This method improves significantly the convergence rate of function values from O(1/k) of the standard gradient method down to O(1/k2)\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$O(1/k^2)$$\end{document}. In this paper, we present two generalized variants of Nesterov’s accelerated proximal gradient method for solving composition convex optimization problems in which the objective function is represented by the sum of a smooth convex function and a nonsmooth convex part. We show that with suitable ways to pick the sequences of parameters, the convergence rate for the function values of this proposed method is actually of order o(1/k2)\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$o(1/k^2)$$\end{document}. Especially, when the objective function is p-uniformly convex for p>2\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$p>2$$\end{document}, the convergence rate is of order Olnk/k2p/(p-2)\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$O\left( \ln k/k^{2p/(p-2)}\right)$$\end{document}, and the convergence is linear if the objective function is strongly convex. By-product, we derive a forward–backward algorithm generalizing the one by Attouch–Peypouquet (SIAM J Optim 26(3):1824–1834, 2016), which produces a convergence sequence with a convergence rate of the function values of order o(1/k2)\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$o(1/k^2)$$\end{document}. Initial computational experiments for solving linear inverse problems with the l1\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$l_1$$\end{document}-regularization demonstrate the capabilities of the proposed algorithms.
引用
收藏
页码:615 / 649
页数:34
相关论文
共 37 条
  • [1] Attouch H(2016)The rate of convergence of Nesterov’s accelerated forward–backward method is actually faster SIAM J. Optim. 26 1824-1834
  • [2] Peypouquet J(2018)Fast convergence of inertial dynamics and algorithms with asymptotic vanishing damping Math. Program. 168 123-175
  • [3] Attouch H(2009)A fast iterative shinkage-thresholding algorithm for linear inverse problems SIAM J. Imag. Sci. 2 183-202
  • [4] Chbani Z(2015)Convex optimization: algorithms and complexity Found. Trends Mach. Learn. 8 231-357
  • [5] Peypouquet J(2018)An optimal first order method based on optimal quadratic averaging SIAM J. Optim. 28 251-271
  • [6] Redont P(2004)An iterative thresholding algorithm for linear inverse problems with a sparsity constraint Commun. Pure Appl. Math. 57 1413-1457
  • [7] Beck A(2007)Gradient projection for sparse reconstruction: application to compressed sensing and other inverse problems IEEE J. Sel. Top. Singal Process. 1 586-597
  • [8] Teboulle M(1964)Convex programming in Hilbert spaces Bull. Am. Math. Soc. 70 709-710
  • [9] Bubeck S(2016)Optimized first-order methods for smooth convex minimization Math. Program. Ser. A 159 81-107
  • [10] Drusvyatskiy D(1966)Constrained minimization problems USSR Comput. Math. Math. Phys. 6 1-50