Replica symmetry breaking in neural networks: a few steps toward rigorous results

被引:19
作者
Agliari, Elena [1 ,2 ]
Albanese, Linda [3 ]
Barra, Adriano [2 ,3 ,4 ]
Ottaviani, Gabriele [5 ]
机构
[1] Sapienza Univ Roma, Dipartimento Matemat Guido Castelnuovo, Rome, Italy
[2] Ist Nazl Matemat Francesco Severi, Rome, Italy
[3] Univ Salento, Dipartimento Matemat & Fis Ennio De Giorgi, Lecce, Italy
[4] Ist Nazl Fis Nucl, Campus Ecotekne, Lecce, Italy
[5] Sapienza Univ Roma, Dipartimento Fis, Rome, Italy
关键词
statistical mechanics; disordered systems; replica symmetry breaking; Hopfield model; rigorous methods; HOPFIELD MODEL; GIBBS-STATES; PATTERNS;
D O I
10.1088/1751-8121/abaf2c
中图分类号
O4 [物理学];
学科分类号
0702 ;
摘要
In this paper we adapt thebroken replica interpolationtechnique (developed by Francesco Guerra to deal with the Sherrington-Kirkpatrick model, namely a pairwise mean-field spin-glass whose couplings are i.i.d. standard Gaussian variables) in order to work also with the Hopfield model (i.e. a pairwise mean-field neural-network whose couplings are drawn according to Hebb's learning rule): this is accomplished by grafting Guerra's telescopic averages on the transport equation technique, recently developed by some of the authors. As an overture, we apply the technique to solve the Sherrington-Kirkpatrick model with i.i.d. Gaussian couplings centered atJ(0)and with finite varianceJ; the meanJ(0)provides a ferromagnetic contribution to be detected in a noisy environment tuned byJ, hence making this model a natural test-case to be investigated before addressing the Hopfield model. For both the models, an explicit expression of their quenched free energy in terms of their natural order parameters is obtained at theKth step (Karbitrary, but finite) of replica-symmetry-breaking. In particular, for the Hopfield model, by assuming that the overlaps respect Parisi's decomposition (following theziqqurat ansatz) and that the Mattis magnetization is self-averaging, we recover previous results obtained via replica-trick by Amit, Crisanti and Gutfreund (1RSB) and by Steffan and Kuhn (2RSB).
引用
收藏
页数:59
相关论文
共 55 条
[1]   Generalized Guerra's interpolation schemes for dense associative neural networks [J].
Agliari, Elena ;
Alemanno, Francesco ;
Barra, Adriano ;
Fachechi, Alberto .
NEURAL NETWORKS, 2020, 128 :254-267
[2]   Dreaming neural networks: rigorous results [J].
Agliari, Elena ;
Alemanno, Francesco ;
Barra, Adriano ;
Fachechi, Alberto .
JOURNAL OF STATISTICAL MECHANICS-THEORY AND EXPERIMENT, 2019,
[3]   Free energies of Boltzmann machines: self-averaging, annealed and replica symmetric approximations in the thermodynamic limit [J].
Agliari, Elena ;
Barra, Adriano ;
Tirozzi, Brunello .
JOURNAL OF STATISTICAL MECHANICS-THEORY AND EXPERIMENT, 2019,
[4]   Non-convex Multi-species Hopfield Models [J].
Agliari, Elena ;
Migliozzi, Danila ;
Tantari, Daniele .
JOURNAL OF STATISTICAL PHYSICS, 2018, 172 (05) :1247-1269
[5]   Neural Networks Retrieving Boolean Patterns in a Sea of Gaussian Ones [J].
Agliari, Elena ;
Barra, Adriano ;
Longo, Chiara ;
Tantari, Daniele .
JOURNAL OF STATISTICAL PHYSICS, 2017, 168 (05) :1085-1104
[6]   Parallel retrieval of correlated patterns: From Hopfield networks to Boltzmann machines [J].
Agliari, Elena ;
Barra, Adriano ;
De Antoni, Andrea ;
Galluzzi, Andrea .
NEURAL NETWORKS, 2013, 38 :52-63
[7]   Interpolating between boolean and extremely high noisy patterns through minimal dense associative memories [J].
Alemanno, Francesco ;
Centonze, Martino ;
Fachechi, Alberto .
JOURNAL OF PHYSICS A-MATHEMATICAL AND THEORETICAL, 2020, 53 (07)
[8]   Deep learning for computational biology [J].
Angermueller, Christof ;
Parnamaa, Tanel ;
Parts, Leopold ;
Stegle, Oliver .
MOLECULAR SYSTEMS BIOLOGY, 2016, 12 (07)
[9]  
[Anonymous], 1987, Spin Glass Theory and Beyond
[10]  
[Anonymous], 2005, Theory of Neural Information Processing Systems