Preconditioned Gradient Method for Data Approximation with Shallow Neural Networks

被引：1

作者：

Vater, Nadja ^{[1
]}

Borzi, Alfio ^{[1
]}

机构：

[1] Univ Wurzburg, Inst Math, Wurzburg, Germany

来源：

MACHINE LEARNING, OPTIMIZATION, AND DATA SCIENCE, LOD 2022, PT II | 2023年 / 13811卷

关键词：

Nonlinear least squares; Regularization; Gradient descent; Preconditioning; Neural networks;

D O I：

10.1007/978-3-031-25891-6_27

中图分类号：

TP18 [人工智能理论];

学科分类号：

081104 ; 0812 ; 0835 ; 1405 ;

摘要：

A preconditioned gradient scheme for the regularized minimization problem arising from the approximation of given data by a shallow neural network is presented. The construction of the preconditioner is based on random normal projections and is adjusted to the specific structure of the regularized problem. The convergence of the preconditioned gradient method is investigated numerically for a synthetic problem with a known local minimizer. The method is also applied to real problems from the Proben1 benchmark set.

引用

页码：357 / 372

页数：16

共 16 条

[1] [Anonymous], 1994, PROBEN1 SET NEURAL N
[2] Broyden C. G., 1973, Journal of the Institute of Mathematics and Its Applications, V12, P223
[3] Crane R., 2021, arXiv
[4] Duchi J, 2011, J MACH LEARN RES, V12, P2121
[5] Goodfellow I, 2016, ADAPT COMPUT MACH LE, P1
[6] Gorbunov E, 2020, PR MACH LEARN RES, V108, P680
[7] Finding Structure with Randomness: Probabilistic Algorithms for Constructing Approximate Matrix Decompositions
Halko, N.
Martinsson, P. G.
Tropp, J. A.
[J]. SIAM REVIEW, 2011, 53 (02) : 217 - 288
[8] Hanke-Bourgeois M, 2009, GRUNDLAGEN NUMERISCH, DOI [10.1007/978-3-8351-9020-7, DOI 10.1007/978-3-8351-9020-7]
[9] HERMAN GT, 1980, J I MATH APPL, V25, P361
[10] Lange S., 2021, ARXIV

← 1 2 →