Restricted Boltzmann machine: Recent advances and mean-field theory*

被引:46
作者
Decelle, Aurelien [1 ,2 ,3 ]
Furtlehner, Cyril [2 ,3 ]
机构
[1] Univ Complutense, Dept Fis Teor 1, Madrid 28040, Spain
[2] Univ Paris Saclay, INRIA Saclay, TAU Team, F-91405 Orsay, France
[3] Univ Paris Saclay, LISN, F-91405 Orsay, France
关键词
restricted Boltzmann machine (RBM); machine learning; statistical physics; STATISTICAL-MECHANICS; NEURAL-NETWORKS; MODEL; STORAGE;
D O I
10.1088/1674-1056/abd160
中图分类号
O4 [物理学];
学科分类号
0702 ;
摘要
This review deals with restricted Boltzmann machine (RBM) under the light of statistical physics. The RBM is a classical family of machine learning (ML) models which played a central role in the development of deep learning. Viewing it as a spin glass model and exhibiting various links with other models of statistical physics, we gather recent results dealing with mean-field theory in this context. First the functioning of the RBM can be analyzed via the phase diagrams obtained for various statistical ensembles of RBM, leading in particular to identify a compositional phase where a small number of features or modes are combined to form complex patterns. Then we discuss recent works either able to devise mean-field based learning algorithms; either able to reproduce generic aspects of the learning process from some ensemble dynamics equations or/and from linear stability arguments.
引用
收藏
页数:24
相关论文
共 89 条
[61]   Mean-field message-passing equations in the Hopfield model and its generalizations [J].
Mezard, Marc .
PHYSICAL REVIEW E, 2017, 95 (02)
[62]  
Nair V., 2010, ICML, P807, DOI DOI 10.5555/3104322.3104425
[63]   Bethe-Peierls approximation and the inverse Ising problem [J].
Nguyen, H. Chau ;
Berg, Johannes .
JOURNAL OF STATISTICAL MECHANICS-THEORY AND EXPERIMENT, 2012,
[64]   Symmetry breaking and training from incomplete data with radial basis Boltzmann machines [J].
Nijman, MJ ;
Kappen, HJ .
INTERNATIONAL JOURNAL OF NEURAL SYSTEMS, 1997, 8 (03) :301-315
[65]   CONVERGENCE CONDITION OF THE TAP EQUATION FOR THE INFINITE-RANGED ISING SPIN-GLASS MODEL [J].
PLEFKA, T .
JOURNAL OF PHYSICS A-MATHEMATICAL AND GENERAL, 1982, 15 (06) :1971-1978
[66]   The Bethe approximation for solving the inverse Ising problem: a comparison with other inference methods [J].
Ricci-Tersenghi, Federico .
JOURNAL OF STATISTICAL MECHANICS-THEORY AND EXPERIMENT, 2012,
[67]   U-Net: Convolutional Networks for Biomedical Image Segmentation [J].
Ronneberger, Olaf ;
Fischer, Philipp ;
Brox, Thomas .
MEDICAL IMAGE COMPUTING AND COMPUTER-ASSISTED INTERVENTION, PT III, 2015, 9351 :234-241
[68]   STATISTICAL-MECHANICS AND PHASE-TRANSITIONS IN CLUSTERING [J].
ROSE, K ;
GUREWITZ, E ;
FOX, GC .
PHYSICAL REVIEW LETTERS, 1990, 65 (08) :945-948
[69]   THE PERCEPTRON - A PROBABILISTIC MODEL FOR INFORMATION-STORAGE AND ORGANIZATION IN THE BRAIN [J].
ROSENBLATT, F .
PSYCHOLOGICAL REVIEW, 1958, 65 (06) :386-408
[70]  
Salakhutdinov R., 2008, P 25 INT C MACH LEAR, V25, P872, DOI DOI 10.1145/1390156.1390266