Functional Central Limit Theorem and Strong Law of Large Numbers for Stochastic Gradient Langevin Dynamics

被引：0
作者：
A. Lovas
M. Rásonyi
机构：
[1] Alfréd Rényi Institute of Mathematics: Renyi Alfred Matematikai Kutatointezet,
[2] Budapest University of Technology and Economics: Budapesti Műszaki és Gazdaságtudományi Egyetem,undefined
[3] Eotvos Lorand University: Eötvös Loránd Tudományegyetem,undefined
来源：
Applied Mathematics & Optimization | 2023年 / 88卷
关键词：
Stochastic gradient descent; Online learning; Functional central limit theorem; Mixing; Markov chains in random environments;
D O I：
暂无
中图分类号：
学科分类号：
摘要：
We study the mixing properties of an important optimization algorithm of machine learning: the stochastic gradient Langevin dynamics (SGLD) with a fixed step size. The data stream is not assumed to be independent hence the SGLD is not a Markov chain, merely a Markov chain in a random environment, which complicates the mathematical treatment considerably. We derive a strong law of large numbers and a functional central limit theorem for SGLD.
引用
收藏
相关论文
共 25 条
[1]  
Barkhagen M(2021)On stochastic gradient Langevin dynamics with dependent data streams in the logconcave case Bernoulli 27 1-33
[2]  
Chau NH(1981)Central limit theorems under weak dependence J. Multivar. Anal. 11 1-16
[3]  
Moulines É(2023)On the ergodicity of certain Markov chains in random environments J. Theor. Probab. 6 1-33
[4]  
Rásonyi M(1996)On the averaged stochastic approximation for linear regression SIAM J. Control Optimiz. 34 31-61
[5]  
Sabanis S(1984)A functional central limit theorem for weakly dependent sequences of random variables Ann. Probab. 12 141-153
[6]  
Zhang Y(2021)On stochastic gradient Langevin dynamics with dependent data streams: the fully nonconvex case SIAM J. Math. Data Sci. 3 959-986
[7]  
Bradley RC(2021)Markov chains in random environment with applications in queuing theory and machine learning Stoch. Processes Appl. 137 294-326
[8]  
Gerencsér B(2016)Consistency and fluctuations for stochastic gradient Langevin dynamics Mach. Learn. Res. 17 193-225
[9]  
Rásonyi M(2016)Exploration of the (non-)asymptotic bias and variance of stochastic gradient Langevin dynamics J. Mach. Learn. Res. 17 1-48
[10]  
Györfi L(undefined)undefined undefined undefined undefined-undefined
123