Restricted Boltzmann machine: Recent advances and mean-field theory*

被引:39
|
作者
Decelle, Aurelien [1 ,2 ,3 ]
Furtlehner, Cyril [2 ,3 ]
机构
[1] Univ Complutense, Dept Fis Teor 1, Madrid 28040, Spain
[2] Univ Paris Saclay, INRIA Saclay, TAU Team, F-91405 Orsay, France
[3] Univ Paris Saclay, LISN, F-91405 Orsay, France
关键词
restricted Boltzmann machine (RBM); machine learning; statistical physics; STATISTICAL-MECHANICS; NEURAL-NETWORKS; MODEL; STORAGE;
D O I
10.1088/1674-1056/abd160
中图分类号
O4 [物理学];
学科分类号
0702 ;
摘要
This review deals with restricted Boltzmann machine (RBM) under the light of statistical physics. The RBM is a classical family of machine learning (ML) models which played a central role in the development of deep learning. Viewing it as a spin glass model and exhibiting various links with other models of statistical physics, we gather recent results dealing with mean-field theory in this context. First the functioning of the RBM can be analyzed via the phase diagrams obtained for various statistical ensembles of RBM, leading in particular to identify a compositional phase where a small number of features or modes are combined to form complex patterns. Then we discuss recent works either able to devise mean-field based learning algorithms; either able to reproduce generic aspects of the learning process from some ensemble dynamics equations or/and from linear stability arguments.
引用
收藏
页数:24
相关论文
共 50 条
  • [1] A dynamical mean-field theory for learning in restricted Boltzmann machines
    Cakmak, Burak
    Opper, Manfred
    JOURNAL OF STATISTICAL MECHANICS-THEORY AND EXPERIMENT, 2020, 2020 (10):
  • [2] Comparing scattering rates from Boltzmann and dynamical mean-field theory
    Wais, M.
    Kaufmann, J.
    Battiato, M.
    Held, K.
    PHYSICAL REVIEW B, 2021, 103 (20)
  • [3] Mean-field theory of social laser
    Alodjants, Alexander P.
    Bazhenov, A. Yu
    Khrennikov, A. Yu
    Bukhanovsky, A., V
    SCIENTIFIC REPORTS, 2022, 12 (01)
  • [4] A Boltzmann approach to mean-field sparse feedback control
    Albi, Giacomo
    Fornasier, Massimo
    Kalise, Dante
    IFAC PAPERSONLINE, 2017, 50 (01): : 2898 - 2903
  • [5] Mean-field theory of inhomogeneous fluids
    Tschopp, S. M.
    Vuijk, H. D.
    Sharma, A.
    Brader, J. M.
    PHYSICAL REVIEW E, 2020, 102 (04)
  • [6] Mean-field theory of superradiant phase transition in complex networks
    Bazhenov, Andrei Yu
    Tsarev, Dmitriy, V
    Alodjants, Alexander P.
    PHYSICAL REVIEW E, 2021, 103 (06)
  • [7] Mean-field theory of meta-learning
    Plewczynski, Dariusz
    JOURNAL OF STATISTICAL MECHANICS-THEORY AND EXPERIMENT, 2009,
  • [8] Mean-field theory for coarsening faceted surfaces
    Norris, Scott A.
    Watson, Stephen J.
    PHYSICAL REVIEW E, 2012, 85 (02):
  • [9] Mean-field theory of failed thermalizing avalanches
    Crowley, P. J. D.
    Chandran, A.
    PHYSICAL REVIEW B, 2022, 106 (18)
  • [10] Density-matrix mean-field theory
    Zhang, Junyi
    Cheng, Zhengqian
    SCIPOST PHYSICS, 2024, 17 (02):