A new gradient method with an optimal stepsize property

被引:22
作者
Dai, YH [1 ]
Yang, XQ
机构
[1] Chinese Acad Sci, Acad Math& Syst Sci, Inst Computat Math & Sci Engn Comp, State Key Lab Sci & Engn Comp, Beijing 100080, Peoples R China
[2] Hong Kong Polytech Univ, Dept Appl Math, Kowloon, Hong Kong, Peoples R China
关键词
linear system; gradient method; steepest descent method; (shifted) power method;
D O I
10.1007/s10589-005-5959-2
中图分类号
C93 [管理学]; O22 [运筹学];
学科分类号
070105 ; 12 ; 1201 ; 1202 ; 120202 ;
摘要
The gradient method for the symmetric positive definite linear system Ax = b is as follows x(k+1) = x(k) - alpha(k)g(k) where g(k) = Ax(k) - b is the residual of the system at x(k) and alpha(k) is the stepsize. The stepsize alpha(k) = 2/lambda(1)+lambda(n) is optimal in the sense that it minimizes the modulus parallel to I - alpha A parallel to(2), where.1 and.n are the minimal and maximal eigenvalues of A respectively. Since lambda(1) and lambda(n) are unknown to users, it is usual that the gradient method with the optimal stepsize is only mentioned in theory. In this paper, we will propose a new stepsize formula which tends to the optimal stepsize as k -> infinity. At the same time, the minimal and maximal eigenvalues,.1 and.n, of A and their corresponding eigenvectors can be obtained.
引用
收藏
页码:73 / 88
页数:16
相关论文
共 12 条
[2]   2-POINT STEP SIZE GRADIENT METHODS [J].
BARZILAI, J ;
BORWEIN, JM .
IMA JOURNAL OF NUMERICAL ANALYSIS, 1988, 8 (01) :141-148
[3]  
Cauchy A.-L., 1847, CR HEBD ACAD SCI, P536, DOI DOI 10.1017/CBO9780511702396.063
[4]   Alternate minimization gradient method [J].
Dai, YH ;
Yuan, YX .
IMA JOURNAL OF NUMERICAL ANALYSIS, 2003, 23 (03) :377-393
[5]   INEXACT AND PRECONDITIONED UZAWA ALGORITHMS FOR SADDLE-POINT PROBLEMS [J].
ELMAN, HC ;
GOLUB, GH .
SIAM JOURNAL ON NUMERICAL ANALYSIS, 1994, 31 (06) :1645-1661
[6]  
FLETCHER R, 2001, BARZILAI BORWEIN MET
[7]   Gradient method with retards and generalizations [J].
Friedlander, A ;
Martínez, JM ;
Molina, B ;
Raydan, M .
SIAM JOURNAL ON NUMERICAL ANALYSIS, 1998, 36 (01) :275-289
[8]  
Golub GH., 1961, Numer Math, V3, P147, DOI DOI 10.1007/BF01386013
[9]   An iterative method with variable relaxation parameters for saddle-point problems [J].
Hu, QY ;
Zou, J .
SIAM JOURNAL ON MATRIX ANALYSIS AND APPLICATIONS, 2001, 23 (02) :317-338
[10]   On the behavior of the gradient norm in the steepest descent method [J].
Nocedal, J ;
Sartenaer, A ;
Zhu, CY .
COMPUTATIONAL OPTIMIZATION AND APPLICATIONS, 2002, 22 (01) :5-35