Functional Central Limit Theorem and Strong Law of Large Numbers for Stochastic Gradient Langevin Dynamics

被引：0

作者：

A. Lovas

M. Rásonyi

机构：

[1] Alfréd Rényi Institute of Mathematics: Renyi Alfred Matematikai Kutatointezet,

[2] Budapest University of Technology and Economics: Budapesti Műszaki és Gazdaságtudományi Egyetem,undefined

[3] Eotvos Lorand University: Eötvös Loránd Tudományegyetem,undefined

来源：

Applied Mathematics & Optimization | 2023年 / 88卷

关键词：

Stochastic gradient descent; Online learning; Functional central limit theorem; Mixing; Markov chains in random environments;

D O I：

暂无

中图分类号：

学科分类号：

摘要：

We study the mixing properties of an important optimization algorithm of machine learning: the stochastic gradient Langevin dynamics (SGLD) with a fixed step size. The data stream is not assumed to be independent hence the SGLD is not a Markov chain, merely a Markov chain in a random environment, which complicates the mathematical treatment considerably. We derive a strong law of large numbers and a functional central limit theorem for SGLD.

引用

共 50 条

[41] A Functional Central Limit Theorem for an ARMA(p, q) Process with Markov Switching
Lee, Oesook
COMMUNICATIONS FOR STATISTICAL APPLICATIONS AND METHODS, 2013, 20 (04) : 339 - 345
[42] A random functional central limit theorem for stationary linear processes generated by martingales
FakhreZakeri, I
Lee, S
STATISTICS & PROBABILITY LETTERS, 1997, 35 (04) : 417 - 422
[43] ON THE FUNCTIONAL CENTRAL LIMIT THEOREM FOR REVERSIBLE MARKOV CHAINS WITH NONLINEAR GROWTH OF THE VARIANCE
Longla, Martial
Peligrad, Costel
Peligrad, Magda
JOURNAL OF APPLIED PROBABILITY, 2012, 49 (04) : 1091 - 1105
[44] Counter-example to the functional central limit theorem for real random fields
El Machkouri, M
Volny, D
ANNALES DE L INSTITUT HENRI POINCARE-PROBABILITES ET STATISTIQUES, 2003, 39 (02): : 325 - 337
[45] A Holderian functional central limit theorem for a multi-indexed summation process
Rackauskas, Alfredas
Suquet, Charles
Zemlys, Vaidotas
STOCHASTIC PROCESSES AND THEIR APPLICATIONS, 2007, 117 (08) : 1137 - 1164
[46] AN INTRODUCTION TO FUNCTIONAL CENTRAL LIMIT-THEOREMS FOR DEPENDENT STOCHASTIC-PROCESSES
ANDREWS, DWK
POLLARD, D
INTERNATIONAL STATISTICAL REVIEW, 1994, 62 (01) : 119 - 132
[47] Strong law of large numbers and mixing for the invariant distributions of measure-valued diffusions
Pinsky, RG
STOCHASTIC PROCESSES AND THEIR APPLICATIONS, 2003, 105 (01) : 117 - 137
[48] Kahane-Khintchine inequalities and functional central limit theorem for stationary random fields
El Machkouri, M
STOCHASTIC PROCESSES AND THEIR APPLICATIONS, 2002, 102 (02) : 285 - 299
[49] Functional central limit theorem for random walks in random environment defined on regular trees
Collevecchio, Andrea
Takei, Masato
Uematsu, Yuma
STOCHASTIC PROCESSES AND THEIR APPLICATIONS, 2020, 130 (08) : 4892 - 4909
[50] Functional central limit theorem for the volume of excursion sets generated by associated random fields
Meschenmoser, D.
Shashkin, A.
STATISTICS & PROBABILITY LETTERS, 2011, 81 (06) : 642 - 646

← 1 2 3 4 5 →