Restricted Boltzmann machine: Recent advances and mean-field theory*

被引:39
|
作者
Decelle, Aurelien [1 ,2 ,3 ]
Furtlehner, Cyril [2 ,3 ]
机构
[1] Univ Complutense, Dept Fis Teor 1, Madrid 28040, Spain
[2] Univ Paris Saclay, INRIA Saclay, TAU Team, F-91405 Orsay, France
[3] Univ Paris Saclay, LISN, F-91405 Orsay, France
关键词
restricted Boltzmann machine (RBM); machine learning; statistical physics; STATISTICAL-MECHANICS; NEURAL-NETWORKS; MODEL; STORAGE;
D O I
10.1088/1674-1056/abd160
中图分类号
O4 [物理学];
学科分类号
0702 ;
摘要
This review deals with restricted Boltzmann machine (RBM) under the light of statistical physics. The RBM is a classical family of machine learning (ML) models which played a central role in the development of deep learning. Viewing it as a spin glass model and exhibiting various links with other models of statistical physics, we gather recent results dealing with mean-field theory in this context. First the functioning of the RBM can be analyzed via the phase diagrams obtained for various statistical ensembles of RBM, leading in particular to identify a compositional phase where a small number of features or modes are combined to form complex patterns. Then we discuss recent works either able to devise mean-field based learning algorithms; either able to reproduce generic aspects of the learning process from some ensemble dynamics equations or/and from linear stability arguments.
引用
收藏
页数:24
相关论文
共 50 条
  • [31] A Removing Redundancy Restricted Boltzmann Machine
    Jiang, Yun
    Xiao, Jize
    Liu, Xi
    Hou, Jinquan
    PROCEEDINGS OF 2018 TENTH INTERNATIONAL CONFERENCE ON ADVANCED COMPUTATIONAL INTELLIGENCE (ICACI), 2018, : 57 - 62
  • [32] Matrix Variate Restricted Boltzmann Machine
    Qi, Guanglei
    Sun, Yanfeng
    Gao, Junbin
    Hu, Yongli
    Li, Jinghua
    2016 INTERNATIONAL JOINT CONFERENCE ON NEURAL NETWORKS (IJCNN), 2016, : 389 - 395
  • [33] Mean-Field Caging in a Random Lorentz Gas
    Biroli, Giulio
    Charbonneau, Patrick
    Hu, Yi
    Ikeda, Harukuni
    Szamel, Grzegorz
    Zamponi, Francesco
    JOURNAL OF PHYSICAL CHEMISTRY B, 2021, 125 (23) : 6244 - 6254
  • [34] Correlated cluster mean-field theory for spin-glass systems
    Zimmer, F. M.
    Schmidt, M.
    Magalhaes, S. G.
    PHYSICAL REVIEW E, 2014, 89 (06):
  • [35] A mean-field theory on the differential capacitance of asymmetric ionic liquid electrolytes
    Han, Yining
    Huang, Shanghui
    Yan, Tianying
    JOURNAL OF PHYSICS-CONDENSED MATTER, 2014, 26 (28)
  • [36] Coupled-cluster impurity solvers for dynamical mean-field theory
    Zhu, Tianyu
    Jimenez-Hoyos, Carlos A.
    McClain, James
    Berkelbach, Timothy C.
    Chan, Garnet Kin-Lic
    PHYSICAL REVIEW B, 2019, 100 (11)
  • [37] What promotes smectic order: Applying mean-field theory to the ends
    King, David A.
    Kamien, Randall D.
    PHYSICAL REVIEW E, 2023, 107 (06)
  • [38] Crowd Management Services in Hajj: A Mean-Field Game Theory Approach
    Nasser, Nidal
    El Ouadrhiri, Ahmed
    El Kamili, Mohamed
    Ali, Asmaa
    Anan, Muhammad
    2019 IEEE WIRELESS COMMUNICATIONS AND NETWORKING CONFERENCE (WCNC), 2019,
  • [39] Anharmonic lattice dynamics from vibrational dynamical mean-field theory
    Shih, Petra
    Berkelbach, Timothy C.
    PHYSICAL REVIEW B, 2022, 106 (14)
  • [40] Rigorous Dynamical Mean-Field Theory for Stochastic Gradient Descent Methods
    Gerbelot, Cedric
    Troiani, Emanuele
    Mignacco, Francesca
    Krzakala, Florent
    Zdeborova, Lenka
    SIAM JOURNAL ON MATHEMATICS OF DATA SCIENCE, 2024, 6 (02): : 400 - 427