The Chaotic Nature of Faster Gradient Descent Methods

被引：0
作者：
Kees van den Doel
Uri Ascher
机构：
[1] University of British Columbia,Department of Computer Science
来源：
Journal of Scientific Computing | 2012年 / 51卷
关键词：
Gradient descent; Iterative methods; Chaos; Dynamical systems;
D O I：
暂无
中图分类号：
学科分类号：
摘要：
The steepest descent method for large linear systems is well-known to often converge very slowly, with the number of iterations required being about the same as that obtained by utilizing a gradient descent method with the best constant step size and growing proportionally to the condition number. Faster gradient descent methods must occasionally resort to significantly larger step sizes, which in turn yields a rather non-monotone decrease pattern in the residual vector norm.
引用
收藏
页码：560 / 581
页数：21
相关论文
共 50 条
[1]  
Akaike H.(1959)On a successive transformation of probability distribution and its application to the analysis of the optimum gradient method Ann. Inst. Stat. Math., Tokyo 11 1-16
[2]  
Ascher U.(2007)Artificial time integration BIT Numer. Math. 47 3-25
[3]  
Huang H.(2009)Gradient descent and fast artificial time integration Modél. Math. Anal. Numér. 43 689-708
[4]  
van den Doel K.(1988)Two point step size gradient methods IMA J. Numer. Anal. 8 141-148
[5]  
Ascher U.(2005)On the asymptotic behaviour of some new gradient methods Math. Program. 103 541-559
[6]  
van den Doel K.(2005)Projected Barzilai-Borwein methods for large-scale box-constrained quadratic programming Numer. Math. 100 21-47
[7]  
Huang H.(2002)R-linear convergence of the Barzilai and Borwein gradient method IMA J. Numer. Anal. 22 1-10
[8]  
Svaiter B.(2003)Alternate minimization gradient method IMA J. Numer. Anal. 23 377-393
[9]  
Barzilai J.(2006)A cyclic Barzilai-Borwein method for unconstrained optimization IMA J. Numer. Anal. 26 604-627
[10]  
Borwein J.(2007)Gradient projection for sparse reconstruction: application to compressed sensing and other inverse problems IEEE J. Special Topics Signal Process. 1 586-598
12345