Restricted Boltzmann machine: Recent advances and mean-field theory*

被引:46
作者
Decelle, Aurelien [1 ,2 ,3 ]
Furtlehner, Cyril [2 ,3 ]
机构
[1] Univ Complutense, Dept Fis Teor 1, Madrid 28040, Spain
[2] Univ Paris Saclay, INRIA Saclay, TAU Team, F-91405 Orsay, France
[3] Univ Paris Saclay, LISN, F-91405 Orsay, France
关键词
restricted Boltzmann machine (RBM); machine learning; statistical physics; STATISTICAL-MECHANICS; NEURAL-NETWORKS; MODEL; STORAGE;
D O I
10.1088/1674-1056/abd160
中图分类号
O4 [物理学];
学科分类号
0702 ;
摘要
This review deals with restricted Boltzmann machine (RBM) under the light of statistical physics. The RBM is a classical family of machine learning (ML) models which played a central role in the development of deep learning. Viewing it as a spin glass model and exhibiting various links with other models of statistical physics, we gather recent results dealing with mean-field theory in this context. First the functioning of the RBM can be analyzed via the phase diagrams obtained for various statistical ensembles of RBM, leading in particular to identify a compositional phase where a small number of features or modes are combined to form complex patterns. Then we discuss recent works either able to devise mean-field based learning algorithms; either able to reproduce generic aspects of the learning process from some ensemble dynamics equations or/and from linear stability arguments.
引用
收藏
页数:24
相关论文
共 89 条
[1]  
ACKLEY DH, 1985, COGNITIVE SCI, V9, P147
[2]   Free energies of Boltzmann machines: self-averaging, annealed and replica symmetric approximations in the thermodynamic limit [J].
Agliari, Elena ;
Barra, Adriano ;
Tirozzi, Brunello .
JOURNAL OF STATISTICAL MECHANICS-THEORY AND EXPERIMENT, 2019,
[3]   Multitasking attractor networks with neuronal threshold noise [J].
Agliari, Elena ;
Barra, Adriano ;
Galluzzi, Andrea ;
Isopi, Marco .
NEURAL NETWORKS, 2014, 49 :19-29
[4]   Multitasking Associative Networks [J].
Agliari, Elena ;
Barra, Adriano ;
Galluzzi, Andrea ;
Guerra, Francesco ;
Moauro, Francesco .
PHYSICAL REVIEW LETTERS, 2012, 109 (26)
[5]   Nonmonotonic generalization bias of gaussian mixture models [J].
Akaho, S ;
Kappen, HJ .
NEURAL COMPUTATION, 2000, 12 (06) :1411-1427
[6]   NEURAL THEORY OF ASSOCIATION AND CONCEPT-FORMATION [J].
AMARI, SI .
BIOLOGICAL CYBERNETICS, 1977, 26 (03) :175-185
[7]   SPIN-GLASS MODELS OF NEURAL NETWORKS [J].
AMIT, DJ ;
GUTFREUND, H .
PHYSICAL REVIEW A, 1985, 32 (02) :1007-1018
[8]   STORING INFINITE NUMBERS OF PATTERNS IN A SPIN-GLASS MODEL OF NEURAL NETWORKS [J].
AMIT, DJ ;
GUTFREUND, H ;
SOMPOLINSKY, H .
PHYSICAL REVIEW LETTERS, 1985, 55 (14) :1530-1533
[9]   STATISTICAL-MECHANICS OF NEURAL NETWORKS NEAR SATURATION [J].
AMIT, DJ ;
GUTFREUND, H ;
SOMPOLINSKY, H .
ANNALS OF PHYSICS, 1987, 173 (01) :30-67
[10]  
[Anonymous], 2010, P 13 INT C ARTIFICIA