On the Convergence of Stochastic Gradient Descent for Linear Inverse Problems in Banach Spaces

被引:4
作者
Jin, Bangti [1 ]
Kereta, Zeljko [2 ]
机构
[1] Chinese Univ Hong Kong, Dept Math, Shatin, Hong Kong, Peoples R China
[2] UCL, Dept Comp Sci, Gower St, London WC1E 6BT, England
基金
英国工程与自然科学研究理事会;
关键词
stochastic gradient descent; Banach spaces; linear inverse problems; convergence rate; regularizing property; almost sure convergence; ILL-POSED PROBLEMS; REGULARIZATION; OPTIMIZATION; PARAMETERS; KACZMARZ; CHOICE;
D O I
10.1137/22M1518542
中图分类号
TP18 [人工智能理论];
学科分类号
081104 ; 0812 ; 0835 ; 1405 ;
摘要
In this work we consider stochastic gradient descent (SGD) for solving linear inverse problems in Banach spaces. SGD and its variants have been established as one of the most successful optimization methods in machine learning, imaging, and signal processing, to name a few. At each iteration SGD uses a single datum, or a small subset of data, resulting in highly scalable methods that are very attractive for large-scale inverse problems. Nonetheless, the theoretical analysis of SGD-based approaches for inverse problems has thus far been largely limited to Euclidean and Hilbert spaces. In this work we present a novel convergence analysis of SGD for linear inverse problems in general Banach spaces: we show the almost sure convergence of the iterates to the minimum norm solution and establish the regularizing property for suitable a priori stopping criteria. Numerical results are also presented to illustrate features of the approach.
引用
收藏
页码:671 / 705
页数:35
相关论文
共 53 条
[1]  
Bogachev V.I., 2007, MEASURE THEORY, V2, DOI 10.1007/978-3-540-34514-5
[2]   Optimization Methods for Large-Scale Machine Learning [J].
Bottou, Leon ;
Curtis, Frank E. ;
Nocedal, Jorge .
SIAM REVIEW, 2018, 60 (02) :223-311
[3]  
Boyd D. W., 1974, Linear Algebra and Its Applications, V9, P95, DOI 10.1016/0024-3795(74)90029-9
[4]   Stable signal recovery from incomplete and inaccurate measurements [J].
Candes, Emmanuel J. ;
Romberg, Justin K. ;
Tao, Terence .
COMMUNICATIONS ON PURE AND APPLIED MATHEMATICS, 2006, 59 (08) :1207-1223
[5]   VARIATIONAL SOURCE CONDITIONS IN Lp-SPACES [J].
Chen, De-Han ;
Yousept, Irwin .
SIAM JOURNAL ON MATHEMATICAL ANALYSIS, 2021, 53 (03) :2863-2889
[6]   Online learning in optical tomography: a stochastic approach [J].
Chen, Ke ;
Li, Qin ;
Liu, Jian-Guo .
INVERSE PROBLEMS, 2018, 34 (07)
[7]   One new strategy for a priori choice of regularizing parameters in Tikhonov's regularization [J].
Cheng, J ;
Yamamoto, M .
INVERSE PROBLEMS, 2000, 16 (04) :L31-L38
[8]   The index function and Tikhonov regularization for ill-posed problems* [J].
Cheng, Jin ;
Hofmann, Bernd ;
Lu, Shuai .
JOURNAL OF COMPUTATIONAL AND APPLIED MATHEMATICS, 2014, 265 :110-119
[9]  
Chidume C, 2009, LECT NOTES MATH, V1965, P1
[10]  
Cioranescu I., 1990, GEOMETRY BANACH SPAC, DOI [10.1007/978-94-009-2121-4, DOI 10.1007/978-94-009-2121-4]