Functional Central Limit Theorem and Strong Law of Large Numbers for Stochastic Gradient Langevin Dynamics

被引：0

作者：

Lovas, A. ^{[1
,2
]}

Rasonyi, M. ^{[1
,3
]}

机构：

[1] Alfred Renyi Inst Math, Budapest, Hungary

[2] Budapest Univ Technol & Econ, Budapest, Hungary

[3] Eotvos Lorand Univ, Budapest, Hungary

来源：

APPLIED MATHEMATICS AND OPTIMIZATION | 2023年 / 88卷 / 03期

关键词：

Stochastic gradient descent; Online learning; Functional central limit theorem; Mixing; Markov chains in random environments; DEPENDENT DATA STREAMS; APPROXIMATION;

D O I：

10.1007/s00245-023-10052-y

中图分类号：

O29 [应用数学];

学科分类号：

070104 ;

摘要：

We study the mixing properties of an important optimization algorithm of machine learning: the stochastic gradient Langevin dynamics (SGLD) with a fixed step size. The data stream is not assumed to be independent hence the SGLD is not a Markov chain, merely a Markov chain in a random environment, which complicates themathematical treatment considerably. We derive a strong law of large numbers and a functional central limit theorem for SGLD.

引用

页数：22

共 50 条

[31] A CENTRAL LIMIT THEOREM FOR STRONG NEAR-EPOCH DEPENDENT RANDOM VARIABLES
LIN ZHENGYAN QIU JIN Department of Mathematics
ChineseAnnalsofMathematics, 2004, (02) : 263 - 274
[32] A central limit theorem for strong near-epoch dependent random variables
Lin, ZY
Qiu, J
CHINESE ANNALS OF MATHEMATICS SERIES B, 2004, 25 (02) : 263 - 274
[33] A CENTRAL LIMIT THEOREM AND HYPOTHESES TESTING FOR RISK-AVERSE STOCHASTIC PROGRAMS
Cuicues, Vincent
Kraetschmer, Volker
Shapiro, Alexander
SIAM JOURNAL ON OPTIMIZATION, 2018, 28 (02) : 1337 - 1366
[34] The functional central limit theorem for Markov-switching GARCH model
Kwon, Dream
Lee, Oesook
ECONOMICS LETTERS, 2024, 238
[35] The strong law of large numbers for L-statistics with dependent data
E. A. Baklanov
Siberian Mathematical Journal, 2006, 47 : 975 - 979
[36] A functional central limit theorem for the level measure of a Gaussian random field
Shashkin, A.
STATISTICS & PROBABILITY LETTERS, 2013, 83 (02) : 637 - 643
[37] The functional central limit theorem and structural change test for the HAR(∞) model
Lee, Oesook
ECONOMICS LETTERS, 2014, 124 (03) : 370 - 373
[38] Functional central limit theorem for double-indexed summation process
Rackauskas A.
Zemlys V.
Lithuanian Mathematical Journal, 2005, 45 (3) : 324 - 333
[39] Langevin Dynamics Based Algorithm e-THεO POULA for Stochastic Optimization Problems with Discontinuous Stochastic Gradient
Lim, Dong-Young
Neufeld, Ariel
Sabanis, Sotirios
Zhang, Ying
MATHEMATICS OF OPERATIONS RESEARCH, 2024,
[40] A functional central limit theorem for the empirical estimator of a semi-Markov kernel
Limnios, N
JOURNAL OF NONPARAMETRIC STATISTICS, 2004, 16 (1-2) : 13 - 18

← 1 2 3 4 5 →