Functional Central Limit Theorem and Strong Law of Large Numbers for Stochastic Gradient Langevin Dynamics

被引：0

作者：

Lovas, A. ^{[1
,2
]}

Rasonyi, M. ^{[1
,3
]}

机构：

[1] Alfred Renyi Inst Math, Budapest, Hungary

[2] Budapest Univ Technol & Econ, Budapest, Hungary

[3] Eotvos Lorand Univ, Budapest, Hungary

来源：

APPLIED MATHEMATICS AND OPTIMIZATION | 2023年 / 88卷 / 03期

关键词：

Stochastic gradient descent; Online learning; Functional central limit theorem; Mixing; Markov chains in random environments; DEPENDENT DATA STREAMS; APPROXIMATION;

D O I：

10.1007/s00245-023-10052-y

中图分类号：

O29 [应用数学];

学科分类号：

070104 ;

摘要：

We study the mixing properties of an important optimization algorithm of machine learning: the stochastic gradient Langevin dynamics (SGLD) with a fixed step size. The data stream is not assumed to be independent hence the SGLD is not a Markov chain, merely a Markov chain in a random environment, which complicates themathematical treatment considerably. We derive a strong law of large numbers and a functional central limit theorem for SGLD.

引用

页数：22

共 50 条

[1] Functional Central Limit Theorem and Strong Law of Large Numbers for Stochastic Gradient Langevin Dynamics
A. Lovas
M. Rásonyi
Applied Mathematics & Optimization, 2023, 88
[2] Functional central limit theorem and Marcinkiewicz strong law of large numbers for Hilbert-valued U-statistics of absolutely regular data
Giraudo, Davide
BRAZILIAN JOURNAL OF PROBABILITY AND STATISTICS, 2024, 38 (02) : 321 - 338
[3] A stochastic model and a functional central limit theorem for information processing in large systems of neurons
Reinhard Höpfner
Klaus Brodda
Journal of Mathematical Biology, 2006, 52 : 439 - 457
[4] A stochastic model and a functional central limit theorem for information processing in large systems of neurons
Höpfner, R
Brodda, K
JOURNAL OF MATHEMATICAL BIOLOGY, 2006, 52 (04) : 439 - 457
[5] The functional central limit theorem for strong near-epoch dependent random variables
QIU Jin 1
2. Information School
ProgressinNaturalScience, 2004, (01) : 10 - 15
[6] The functional central limit theorem for strong near-epoch dependent random variables
Qiu, J
Lin, ZY
PROGRESS IN NATURAL SCIENCE-MATERIALS INTERNATIONAL, 2004, 14 (01) : 9 - 14
[7] A functional central limit theorem for empirical processes under a strong mixing condition
Tone C.
Statistical Inference for Stochastic Processes, 2012, 15 (2) : 177 - 192
[8] Strong Law of Large Numbers for α-Mixing Sequences
Yang Wenquan
PROCEEDINGS OF THE 2014 INTERNATIONAL CONFERENCE ON MECHATRONICS, ELECTRONIC, INDUSTRIAL AND CONTROL ENGINEERING, 2014, 5 : 440 - 443
[9] Discretization of the Ergodic Functional Central Limit Theorem
Pages, Gilles
Rey, Clement
JOURNAL OF THEORETICAL PROBABILITY, 2023, 36 (01) : 1 - 44
[10] A functional central limit theorem for regression models
Bischoff, W
ANNALS OF STATISTICS, 1998, 26 (04) : 1398 - 1410

← 1 2 3 4 5 →