Functional Central Limit Theorem and Strong Law of Large Numbers for Stochastic Gradient Langevin Dynamics

被引：0

作者：

Lovas, A. ^{[1
,2
]}

Rasonyi, M. ^{[1
,3
]}

机构：

[1] Alfred Renyi Inst Math, Budapest, Hungary

[2] Budapest Univ Technol & Econ, Budapest, Hungary

[3] Eotvos Lorand Univ, Budapest, Hungary

来源：

APPLIED MATHEMATICS AND OPTIMIZATION | 2023年 / 88卷 / 03期

关键词：

Stochastic gradient descent; Online learning; Functional central limit theorem; Mixing; Markov chains in random environments; DEPENDENT DATA STREAMS; APPROXIMATION;

D O I：

10.1007/s00245-023-10052-y

中图分类号：

O29 [应用数学];

学科分类号：

070104 ;

摘要：

We study the mixing properties of an important optimization algorithm of machine learning: the stochastic gradient Langevin dynamics (SGLD) with a fixed step size. The data stream is not assumed to be independent hence the SGLD is not a Markov chain, merely a Markov chain in a random environment, which complicates themathematical treatment considerably. We derive a strong law of large numbers and a functional central limit theorem for SGLD.

引用

页数：22

共 50 条

[21] The functional central limit theorem for ARMA GARCH processes
Lee, O.
ECONOMICS LETTERS, 2013, 121 (03) : 432 - 435
[22] Exploration of the (Non-)Asymptotic Bias and Variance of Stochastic Gradient Langevin Dynamics
Vollmer, Sebastian J.
Zygalakis, Konstantinos C.
Teh, Yee Whye
JOURNAL OF MACHINE LEARNING RESEARCH, 2016, 17 : 1 - 45
[23] A FUNCTIONAL CENTRAL LIMIT THEOREM FOR SPATIAL BIRTH AND DEATH PROCESSES
Qi, Xin
ADVANCES IN APPLIED PROBABILITY, 2008, 40 (03) : 759 - 797
[24] A functional central limit theorem for Hilbert-valued martingales
Lavrentyev V.V.
Nazarov L.V.
Lobachevskii Journal of Mathematics, 2016, 37 (2) : 138 - 145
[25] A functional central limit theorem for diffusions on periodic submanifolds of ℝN
Brice Franke
Probability Theory and Related Fields, 2005, 133 : 236 - 244
[26] A functional central limit theorem for diffusions on periodic submanifolds of RN
Franke, B
PROBABILITY THEORY AND RELATED FIELDS, 2005, 133 (02) : 236 - 244
[27] A Functional Central Limit Theorem for the Becker-Doring Model
Sun, Wen
JOURNAL OF STATISTICAL PHYSICS, 2018, 171 (01) : 145 - 165
[28] A Functional Central Limit Theorem for the Becker–Döring Model
Wen Sun
Journal of Statistical Physics, 2018, 171 : 145 - 165
[29] A Contour Stochastic Gradient Langevin Dynamics Algorithm for Simulations of Multi-modal Distributions
Deng, Wei
Lin, Guang
Liang, Faming
ADVANCES IN NEURAL INFORMATION PROCESSING SYSTEMS (NEURIPS 2020), 2020, 33
[30] Central limit theorems for stochastic gradient descent with averaging for stable manifolds*
Dereich, Steffen
Kassing, Sebastian
ELECTRONIC JOURNAL OF PROBABILITY, 2023, 28

← 1 2 3 4 5 →